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Abstract 

Many decision problems on security protocols can be reduced to solving 
so-called intruder constraints in Dolev Yao model. Most constraint solving 
procedures for protocol security rely on two properties of constraint systems 
called monotonicity and variable- origination. In this work we relax these 
restrictions by giving a decision procedure for solving general intruder con- 
straints (that do not have these properties) that stays in NP. Our result 
extends a first work by L. Mazare in several directions: we allow non- atomic 
keys, and an associative, commutative and idempotent symbol (for modeling 
sets). We also discuss several new appUcations of the results. 

Keywords: ACI, deducibility constraints, Dolev- Yao deduction system, 
multiple intruders, security. 



1. Introduction 

Detecting flaws in security protocol specifications under the perfect cryp- 
tography assumption in Dolev- Yao intruder model is an approach that has 
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been extensively investigated in recent years [1, 2, 3, 4]. In particular symbolic 
constraint solving has proved to be a very successful approach in the area. It 
amounts to express the possibility of mounting an attack, e.g. the derivation 
of a secret, as a list of steps where for each step some message has to be 
derived from the current intruder knowledge. These steps correspond in 
general to the progression of the protocol execution, up to the last one which 
is the secret derivation. 

Enriching standard Dolev-Yao intruder model with different equational 
theories [5, 6] like exclusive OR, modular exponentiation, Abelian groups, etc. 
[7, 8, 9] helps to find flaws that could not be detected considering free symbols 
only. A particularly useful theory is the theory of an ACI operator (that is 
associative commutative and idempotent) since it allows one to express sets 
in cryptographic protocols. 

Up to one exception [10, 11], all proposed algorithms rely on two strong 
assumptions about the constraints to be processed: knowledge monotonicity 
and variable origination. Constraints satisfying this hypothesis are called 
well-formed constraints in the literature and they are not restrictive as these 
conditions hold when handling standard security problems with a single Dolev- 
Yao intruder. However, we will see that in some situations it can be quite useful 
to relax these hypotheses and consider general constraints, that is constraints 
without the restrictions above. General constraints naturally occur when 
considering security problems involving several non-communicating Dolev- 
Yao intruders (see § 2.1). Remark that if intruders can communicate during 
protocol execution, the model becomes attack-equivalent to one with a unique 
Dolev-Yao intruder [12]. 

1.1. Contributions of the paper 

First, we will show that as for the standard case, in this more general 
framework it is still possible to derive an NP decision procedure for detecting 
attacks on a bounded number of protocol sessions (Sections 5, 4). Second, 
our result extends previous ones by allowing non-atomic keys and the usage 
of an associative commutative idempotent operator (Sections 3, 4) that can 
be used for instance to model sets of nodes in XML document (see § 2.2). 
Third, we will remark that the satisfiability procedure we obtain for general 
constraints is a non trivial extension of the one for well-formed constraints 
by showing that this procedure cannot be extended to handle operators with 
subterm convergent theories since satisfiability gets undecidable in this case 
(Appendix A). On the other hand it is known that satisfiability remains 
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decidable for the standard case of well-formed constraints with the same 
operator properties [13]. Finally we will sketch the potential applications of 
our results (Section 2). 

1.2. Related works 

The decision procedure for satisfiability of well-formed constraint systems 
can be used to decide the insecurity of cryptographic protocols with a bounded 
number of sessions [14]. In this domain, several works deviated from the 
perfect cryptography assumption and started to consider algebraic properties 
of functional symbols. For example properties of XOR operator and expo- 
nentiation were considered in [15, 8, 16, 17] and together with homomorphic 
symbol in [18]. Some algebraic properties (like associative and commutative 
symbol) make the insecurity problem undecidable [19]. 

All the works mentioned above consider systems of constraints with two 
restrictions namely knowledge monotonicity (the left-hand side of a constraint 
representing the current knowledge of the intruder is included into the left- 
hand side of the next one) and variable origination (variable appears first in 
the right-hand side of some constraint): this limitation is not impeding the 
solution of usual protocol insecurity problems since the constraints generated 
with an active Dolev-Yao intruder are of the required type. An attempt to 
swerve from well-formed constraints was made by Mazare [10]. He considered 
"quasi well-formed" constraint systems by partially relaxing the knowledge 
monotonicity. Later, in his thesis [11], he raised a similar decidability problem, 
but now for general constraint systems. He succeeded to find a decision 
procedure for satisfiability of general constraint systems with the restriction 
that keys used for encryption are atomic. However to our knowledge no 
extension of Dolev-Yao deduction system to non-atomic key or to algebraic 
properties has been shown decidable for general constraint systems. Moreover, 
satisfiability of well-formed constraints with ACI theory was not considered 
before. 

2. Motivating examples 

2.1. Protocol analysis with several intruders 

In the domain of security protocol analysis Dolev-Yao model is widely 
used in spite of its limitations. We propose here to consider instead of a 
powerful Dolev-Yao intruder that controls the whole network, several non 
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Figure 1: Untrusted routers 



communicating Dolev-Yao intruders with smaller controlled domains. We 
give below an application of this model. 

Suppose several agents {A, B . . . , see Figure 1) execute a message exchange 
protocol (every agent has a finite list of actions in a send/receive format that 
is known to everybody). Due to their (long distance) layout they have to 
transmit data through routers (1, 2, 3 ... ). The routing tables of all honest 
routers/agents are static (messages follow always the same path). Some 
routers (2, 5, 7) may be compromised: an intruder managed to install a 
device controlling input and output of the router or implanted there his 
malicious code. A message circulated via such an untrusted channel (e.g. 
DB) is consumed by the corresponding compromised device {local intruder) 
(7) thereby increasing his knowledge. Moreover, a local intruder can forge 
and emit to an endpoint (C, B, D) of any channel he controls (-B-D, DB, DC) 
any message he can build using the content of his memory and some available 
transformations specified by a deduction system. Because of the network 
topology malicious routers have no means to communicate (there is no links 
between them, neither direct nor via other routers), but at some point 
the intruder can gather the knowledge of all the compromised routers (by 
physically collecting devices or reading their memory). 
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In this framework the security problem is to know whether it is possible 
to initially give instructions to compromised routers (e.g. by reprogramming 
malicious devices) to force such an execution that honest agents (that strictly 
follow their list of actions) will reveal some secret data to the intruder (i.e. 
intruder can build this data from the gathered at the end knowledge of all 
local intruders). 

2.1.1. Formalizing the coordinated attack problem 

To formalize the problem we introduce some notations and definitions 
that are more detailed in Subsection 3.1. 

Messages. We consider first-order terms built from a set of function sym- 
bols (such as encryption, pairing, etc.), a set of constants A (representing 
elementary pieces of data: texts, public keys, names of agents, etc. also called 
atoms) and a set of variables X. Let T be the set of all possible terms. For a 
term t we write Vars (t) the set of all variables in t (see Def. 3.9). A term t is 
a ground term, if Vars (t) = 0. The set of ground terms is denoted by Tg. We 
assume that terms are interpreted modulo an equational theory and that we 
can compute for every term t a unique normal form denoted by '"t"' modulo 
this equational theory. (We will focus later on the special case where we 
have a function symbol ■ and the theory is generated by the commutativity, 
associativity and idempotency of-). A term t is normalized if t = ^f^. Two 
terms p and q are equivalent, if ^p~^ = rgi. Given a set of terms T we define 
rT"! = {rfi : t G T}. More details are given in Section 3. 

We define a substitution cr = {xi ti, . . . ,Xk ^ tk} (where Xi E X and 
ti G T) to be the mapping a : T ^ T, such that ta is a term obtained by 
replacing, for all i, each occurrence of variable by the corresponding term 
ti. The set of variables {xi, . . . , Xk} is called the domain of a and denoted by 
dom (cr). If T C 7", then by definition Ta = {ta : t G T}. A substitution a is 
ground if for any i G {1, . . . , fc}, is ground. We will say that the substitution 
a is normalized, if for all x G dom (cr), xa is normalized. 

Agents. We will call communicating parties agents. Every agent is identified 
by its name. We denote a set of agent names as A. 

Channels. Any two agents a and b communicate through a channel denoted 
as a ^ 6. We will suppose, that channels are directed. The set of all channels 
is denoted as C. A channel supports a queue of messages: for example, if 
a sends sequentially two messages to b (via channel a ^ b), then b cannot 
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process the second message before the first one; the sent messages are stored 
in queue to be processed in order of arrival. 



Agents behavior. We define a protocol session PS = {(oj, li)}^^^ ^ as a finite 
set of pairs of an agent name and a finite list of actions to be executed by this 
agent^. We also suppose that Vars (/j) fl Vars (Ij) = 0, for all i j (where 
Vars (■) is naturally extended on lists of actions). 

Every action is of receiving type ?jr or sending type \ts, where 

• / is an agent name, whom a message is to be received from; 

• r is a term (a template for the message) expected to be received from /; 

• t is an agent name, whom the message is expected to be sent to; 

• s is a term (a template for the message) to be sent to t. 

Let us consider any agent a G A participating in the protocol session PS 
and let (a, {pi}j=i^...^fc/ ^ P^- 



Case 1. If pi =?/i?"i then the first action agent a can do, is to accept a 
message m, admittedly from agent /i on channel /i ^ a, matching the pattern 
ri, i.e. such that '"ricr"' = '"m"' for some substitution a. Agent is blocked 
(does not execute any other actions) by awaiting a message. If a receives a 
message that does not match the expected pattern, then a terminates his 
participation in PS. Note that no notification is sent to the sender, thus a 
sender continues his execution^. Once a has received message m matching the 
pattern ri with substitution a, he instantiate Vars (ri) with a and execute 
his remaining actions using these values, i.e. a moves to a state where the 
list of actions to be executed is {piO"}^^2 fc' with 



^For simplicity, we suppose that for a protocol session, one agent cannot have more 
than one list of actions to execute, but this restriction can be relaxed. 

way to model another behavior, is to explicitly provide for every sending a succedent 
receive of an acknowledge message and for every receive a succedent send of an acknowledge 
message. 




?/(m), if p =?/r; 
\t{sa), if p =\ts. 
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We will say that an action p is ground, if p =?/r and r is a ground term; or 

p =\ts and s is ground. 

Case 2. If pi is It^si then the first action of agent a is sending message Si 
to agent ti (i.e. putting it to channel a ti) and then, moving to a state 
where {pi}^^2 k ^® executed. 

We suppose that agents cannot have a sending pattern that contains 
variables not instantiated before, i.e. for any {a, pi. ■ ■ ■ .pk^) G PS if pi —\tS 
then for any variable x G Vars (s) there exists j < i such that pj =?/r and 
X G Vars (r). 

Intruder model. We assume that some communication channels are controlled 
by N local intruders {li}^^^ and there is no channel controlled by more 
than one intruders. We introduce an intruders layout represented by a 
function t:Ci->-IU{0} mapping every channel to the local intruder that 
controls it if there is one, to otherwise. 

Every intruder / is given some initial knowledge Kj that is a set of ground 
terms. Once an agent sends a message via a channel controlled by intruder, 
the intruder reads it and blocks it. Reading the message means extending 
intruder's current knowledge with this message. An intruder controlling a 
channel can generate a message from his knowledge using deduction rules and 
send it to its endpoint. 

We now specify the intruder capabilities: 

Definition 2.1. A rule is a tuple of terms written as si, . . . , ^ s, where 
Si, . . . ,Sk,s are terms. A deduction system V is a set of rules. 

Prom now to the end of this section rules are assumed to belong to a fixed 
deduction system V. 

Definition 2.2. A ground instance of a rule d = si, . . . , — >■ s is a rule 
Z = /i, . . . , /fc — >■ r where /i, . . . , Z^, r are ground terms and there exists a — 
ground substitution, such that /j = SjO", for i — 1, . . . , k and r = sa. We call 
a ground instance of a rule a ground rule. 

Given two sets of ground terms E, F and a rule I ^ r,we write E -^i^r F 
iS F = E U {r} and / C E, where / is a set of terms. We write E ^ F iS 
there exists rule / r such that E -^i^r F- 

Definition 2.3. A derivation D of length n > is a sequence of finite sets 
of ground terms Eq, Ei, . . . , En such that Eq ^ Ei ^ • • ■ ^ En, where 
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Ei = Ei_i U {tj} , Vi = {1, . . . , n}. A term t is derivable from a set of terms 
E iff there exists a derivation D = Eq, . . . , En such that Eq = E and t & En- 
Pi. set of terms T is derivable from E, iff every t G T is derivable from E. We 
denote Der [E) set of terms derivable from E. 

Local intruder I can send a message m, if m G Der (-ft'/), where i^/ is a 
current knowledge of intruder J. 

Protocol session execution. Now,we can present a course of a protocol execu- 
tion. We first introduce a notion of symbolic execution, where data exchanged 
among the agents and intruders are not instantiated and represented as (pos- 
sibly non-ground) terms. This execution is constrained by some conditions. 
Whenever these conditions are satisfied by an appropriate ground instanti- 
ation of variables, we obtain a concrete execution, or simply an execution. 
These conditions are defined by constraint systems: 

Definition 2.4. Let Ehe a, set of terms and t be a term, we define the couple 
(-E, t) denoted > t to be a constraint. A constraint system is a set 

S = {E,t> 

where n is an integer and Ei > tj is a constraint for i G {1, . . . , n}. 

We extend the definition of Vars (■) to constraint system iS in a natural 
way. We say that S is normalized, if every term in S is normalized. By ^S^ 
we will denote a constraint system {^Ep \> ^tp}^^^ ^. 

Definition 2.5. A ground substitution a is a model of constraint E \> t (or 
a satisfies this constraint), if ^ta~^ G Der {^Ea~^). A ground substitution a is 
a model of a constraint system S, if it satisfies all the constraints of S and 
dom (cr) = Vars (S). 

Definition 2.6. A configuration 11 of a protocol session is a quadruple 

{PS,IC, Q,S), where /C = {{Ii,Ki)}^ 

=1 ... N represents current knowledges of 
intruders, and Q = {{c,mc)}^^^ is a configuration of channels: for every 
channel c queue of messages rric is given. 

Transitions on configurations are defined in Table'^ 1 and will be explained 
later. Transitions are written in form IIi '^°"'^> 112 and state that configuration 
III can evolve to a new configuration 112 if condition cond is satisfied. 



^1+1 represents the union of two disjoint sets: B ~ AU B iS A B = 9. 
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1. {{{a, (?/r)./,)} W PS, {{I, K)}iS}C, Q, S) ^^^^^ 
({(a, Q} U PS, {{I, K)} U}C,Q,SU{K> r}) 

2. {{{a, W PS, {{I, K)} W /C, Q, 5) ^^^^ 
{{{a, Q} U P5, {(/, U s)} U /C, Q, S) 

3. ({(a, (lis)./,)} W P5,/C,{(a - t,m,^,)} W Q,S) 
({(a, U PS, /C, {(a - t, m^^^.s)} U Q, S) 

4. ({(a, (?/r)i,)} W P^, /C, {(/ - a, s.m^^,)} W Q, 5) ^1^^^ 

({(g, Ig)} U Pg, /C, {(/ ^ a, nif^a)} UQ,SU {{enc {s, k)} > enc (r, k)}) 



Table 1: Configuration transitions 



Definition 2.7. A symbolic execution Ep^ of protocol session PS (with 
intruders layout l) is a sequence of configurations obtained by application of 
transitions to the initial configuration (PS', {(/, K'^)}j^j , {{c, 0)}cgc ' '^)- 

For a substitution a and a configuration 11 = 
({(«i> ^i)}i=i,...,fc > ^*)}i=i,...,iv ' {(c, "^c)}cec ' ^ ^i}i=i,...,n) we define Ha 

as (^{{tti, , ^iC^)K=i,...,^ , {(C, mcOr)}^gc ' i^^^ ^ ^i^} i=l,...,n) ' 

where substitutions are applied to lists elementwise. 

Definition 2.8. An execution Eps = {C'iO"}j=i ^ instance of a sym- 
boUc execution {Ci}-^^ ^ (where Cj = {PSi, /Cj, Qi,Si)) such that all terms 
of Cjcr are ground and Sm is satisfied by a. 

Now we describe the transitions of Table 1. Transition f expresses the 
possibility of intruder / controlling channel / ^ a to impersonate / and send 
to a some message compliant with the expected by a pattern r, if the current 
knowledge of / allows it. An intruder can also intercept messages sent on 
the channel that he controls (Transition 2). A message sent by an agent on 
the channel free from intruders is put to the end of the queue of this channel 
(Transition 3). Transition 4 represents the reading of a message from the 
queue of the channel. 

Let us explain where the constraint {enc {s, k)} > enc (r, k) comes from 
in the last transition. Agent a expects to read a message from the channel 
compatible with the pattern r. The first (possibly not yet instantiated) 
message in the queue is s. Thus, r and s must be unifiable (modulo considered 
equational theory), and even equivalent when we consider ground instances 
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of the symbolic executions. Since we will be interested only in concrete 
executions, but not symbolic, we can use this constraint to express equivalence 
between r and s (Lemma 1). 

Lemma 1. For terms ti, t2 and substitution o, ^t\a'^ = ^t2<y'^ is true ijf a 
is a model of {enc (ti, k)} > enc (^2, k) for any term k, i.e. ^tia^ = '~t2cr~i iff 
renc (ti, k) e Der ({^enc (t2, k) a^}) . 

Offline communication. At some point the current knowledge of all local 
intruders can be shared to derive a secret which probably they cannot deduce 
separately. In some cases these offline interactions are time-consuming and 
may be detected. Therefore we consider reasonable that in the intruder 
strategy modelling they take place after the protocol is over. 

Coordinated attack problem. Now we can formally state the problem. 
Input: A finite set of agents A, a protocol session PS = f,, a 

set of intruders I = {li}^^^ jy each with initial knowledge Kj,, an intruder 
layout i and some sensitive data given as a finite set of ground terms S. 
Output: s G S and an execution Eps of protocol session PS with its last 
configuration {PS,}C, Q,S) such that s G Der ^|J^^^ ^-^^^ 7^"/^ 

2.1.2. Solving the problem 
We proceeds as follows: 

1. Guess a sensitive datum s from 5*. 

2. Guess a symbolic execution Epg of some length < ^ length(Z) < oo. 

{a,l)£PS 

3. For the last configuration {PS, {{Ki, , Q, 5) of Epg, if constraint 
system S U |U(i<'j i)^k^i ^ satisfiable with some a, then the 
protocol session is insecure and we return Eps = Epgcr. 

We will show in Sections 3 and 4 that the satisfiability of constraint 
systems is in NP in the case of DY+ACI deduction theory. 

2.2. Attack exploiting XML format of messages 

Here we show how to model (using our formalism) attacks based on an 
XML-representation of messages. A different technique to handle this kind of 
attacks was presented in [20]. 



10 



Client 



Shop Interface 
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+ Delivery 



ItemID,Cheque,Address, Comments ^ 

Check the price of item 
and use the cheque 

ItemID,Addres s , Comments 



Send ordered item to given Address 




Figure 2: Ordering item scenario 



We consider an e-shop that accepts e-cheques, and we suppose that it is 
presented by a Web Service using SOAP protocol for exchanging messages. 
It consists of two services: 

• the first exposes the list of goods for sale with their prices and process 
the orders by accepting payment, 

• the second is a delivery service; it receives information from the first 
one about successfully paid orders, and sends the ordered goods to the 
buyer. 

A simple scenario for ordering item is shown in Figure 2. First, a client 
sends an order using e-shop interface that consists of an item identifier, e- 
cheque, delivery address and some comments. Then, the first service of the 
e-shop checks whether the price of the ordered item corresponds to the received 
cheque. If it does, the service consumes the cheque and resends the order to 
the stock/delivery service (without the used e-cheque). Stock and delivery 
service prepare a parcel with ordered item and send it to given address. The 
comment is automatically printed on the parcel to give some information to 
the postman about, for example, delivery time or access instructions. 

Suppose, Alice has an e-cheque for 5€. She selected a simple pen (with 
ItemID simple) to buy, but she liked very much a more expensive gilded one 
(with ItemID gilded). Can we help Alice to get what she wants for what she 
has? 

11 




Let us formalize the behaviour of scenario players (terms, normalization 
function and deduction system are defined as in § 3.1.1 except that we will 
write (ti ■ . . . ■ tn) instead of ■ {{ti, . . . , t„})). Identifiers starting from a capital 
letter are considered as variables; numbers and identifier starting from lower- 
case letter are considered as constants. We model a delivery of item with 
some ItemlD to address Address with comments Comments by the following 
message: sig {{ItemID ■ Address ■ Comment s),pTiv (kg)) — a message signed 
by e-shop, where kg its public key, such that no one can produce this message 
except the shop. We abstract away from the procedure of checking price of 
the item and will suppose, that Shop Interface expects 5€ e-cheque for Item 
"simp/e". For simplicity we assume only two items. 

We will use notation for sending and receiving as in § 2.1. 

For Shop Interface we have: 

"^ciientisimple ■ chequed ■ lAddr ■ IComm); 
! Deiivery{simple ■ I Addr ■ IComm). 

For Shop Stock/Delivery we have: 

Inter face{DItemID ■ D Addr ■ DComm); 
^■Client sig {{DItemID ■ DAddr ■ DComm), phv (kg)) . 

Alice initially has: 

simple, gilded: identifiers of items; 

chequed: an e-cheque for 5€; 

addr: her address; 

cmnts: residence digital code; 

kg: a public key of the shop. 

Now we build a mixed constraint system (derivation constraints and 
equations) to know, whether Alice can do what she wants: 



' {gilded, simple, chequed, addr, cmnts, kg} \> 

{simple ■ chequed ■ I Addr ■ IComm) 
{simple ■ I Addr ■ IComm) =Aci{DItemID ■ DAddr ■ DComm) 
{gilded, simple, chequed, addr, cmnts, kg, 
sig {{DItemID ■ DAddr ■ DComm), phv {kg))} > 

sig {{gilded ■ addr ■ DComm), phv {kg)) 



(1) 
(2) 



(3) 
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Constraint (1) shows, that Ahce can construct a message expected by 
the shop from a chent. Constraint (2) represents a request from the first 
to the second service of the shop: left-hand side is a message sent by the 
interface service, and right-hand side is a message expected by stock/delivery 
subservice. The last constraint shows, that from the received values Alice can 
build a message that models a delivery of item with ItemID gilded. 

To solve it, we first get rid of syntactic equations by applying most general 
unifier; and then of equations modulo ACI (ti =aci ^2 is equivalent to 
rti"! = rtg"!) by encoding them into deduction rules (as it was done in § 2.1.2). 

Then, one of the solutions is: 

lAddr h^addr IComm ^{gilded ■ cmnts) 

DItemID \-^gilded DAddr ^addr 

DComm ^{simple ■ cmnts) 

Prom this solution we see, that Alice can send a not well-formed comments 
(that presents two XML-nodes), and Delivery service parser can choose an 
entry with ID gilded. An attack-request can look like this: 

<ItemID>simple</ItemID> 
<Cheque>cheque5</ Cheque> 
<Address>addr</ Address> 
<Comments>cmnts</ Comments> 
<ItemID>gilded</ItemID> 

The parser of the first service can return value of the first occurrence of 
ItemID: <ItemID>simple</ItemID>. But the parser of the second one can 
return <ItemID>gilded</ItemID>. 

This attack is possible, if Alice constructs a request "by hand", but a 
similar attack is probably feasible using XML-injection: Alice when filling a 
request form enters instead of her comments the following string: 

cmnts</ Comments> 

<ItemID>gilded</ItemIDxComments> 

and in the resulting request we get: 

<ItemID>simple</ItemID> 
<Cheque>cheque5</ Cheque> 
<Address>addr</ Address> 
<Comments>cmnts</ Comments> 
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<ItemID>gilded</ItemIDxComments> 
</ Comments> 

This kind of XML-injection attacks was described in [21]. 

3. Satisfiability of general DY+ACI constraint systems 

In Section 2 we reduced the problem of protocol insecurity in presence of 
several intruders to solving a system of deducibility constraints. In this section 
we present a decision procedure for a constraint system where Dolev-Yao 
deduction system is extended by an associative-commutative-idempotent sym- 
bol (DY+ACI). We consider operators for pairing, symmetric and asymmetric 
encryptions, decryption, signature and an ACI operator that will be used as 
a set constructor. 

As for the proof structure, after introducing the formal notations, the 
main steps to show the decidability follows: 

1. We present an algorithm for solving a ground derivability in DY+ACI 
model. 

2. We prove, that the normalization does not change satisfiability: either 
we normalize a model or a constraint system. 

3. We show existence of a conservative solution of satisfiable constraint 
system: a substitution a that sends a variable to an ACI-set of quasi- 
subterms of the constraint system instantiated with a together with 
priv-ed atoms of the constraint system; 

4. We give a bound on size of a conservative solution, and, as consequence, 
we obtain decidability. 

3.1. Formal introduction to the problem 
3.1.1. Terms and notions 

Definition 3.1. Terms are defined according to the following grammar: 

term ::= variable \ atom \ pair [term, term) \ 

enc {term, term) \ -{tlist) \ phv {Keys) \ 
aenc {term, Keys) \ sig {term, phv {Keys)) 

Keys :: = variable \ atom 

tlist ::= term \ term, tlist 
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where atom e A and variable G X. We denote T{A, X) the set of all terms 
over a set of atoms A and a set of variables X. For short, we write T instead 



By sig (p, priv (a)) we mean a signature of message p with private key 
priv (a) We do not assume that one can retrieve the message itself from the 
signature. 

Note that we do allow complex keys for symmetric encryption only. As a 
consequence, we have to introduce a condition on substitution applications: 
substitution a cannot be applied to the term t, if after replacing the resulting 
entity is not a term (for example, we cannot apply a — {x pair (a, b)} to 
the term aenc {a,x)). 

We denote a term on i-th. position of a list L as L[i]. Then t E L is a 
shortcut for 3i : t — L[i]. We also define two binary relations C and on 
lists as follows: Li C L2 if and only if any t G Li implies t G L2; Li L2 if 
and only if Li C L2 and L2 C Li, and naturally extend them if Li or L2 is a 
set. 

Definition 3.2. We consider symbol • to be associative, commutative, idem- 
potent (shortly, AC I). 

We will use bin throughout the paper as a generalization of all binary 
operators: bin G {enc, aenc, pair, sig}. 

Definition 3.3. For every term t E T we define its root symbol by 



Definition 3.4. For any term t eT we define its set of elements by: 



We extend elems() to sets of terms or lists of terms T by elems(T') = 
(Jjgy elems {t). 

Example 1. For term t = ■ ({a, • ({&, a, pair (a, &)}) , pair (• ({&, 6}) , a)}) set 
of its elements is elems {t) — {a, b, pair (• ({6, b}) , a) , pair (a, b)}. 



of r{A,X). 




elems (t) 



UpeL elems (p) if t = ■ (L) ; 
{t} , otherwise. 
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Definition 3.5. Let -< be a strict total order on T, such that comparing can 
be done in polynomial time. 

Definition 3.6. The cardinality of a set P is denoted by \P\. 

Definition 3.7. The normal form of a term t (denoted by '"t"') is recursively 
defined by: 

• rbin(ti,t2)^ = bin(rtin^rt2n) 

• Tpriv (t)"i = priv ('"t"') 

{V) , if |r-elems > 1 and L' ^ rglems (L)^ 
r- (L)^ = { and for all i < j, L'\i] -< L'[j]; 

t', if relems(L)^ = {t'} 

where for set of terms T, '~T~^ = {rfi : t E T}. 

We can show easily that two terms are congruent modulo the ACI proper- 
ties of "." iff they have the same normal form. Other properties are stated in 
Lemma 4. 

Example 2. Referring to Example 1 for the value of term t, we have '~t^ = 
■ ({a, b, pair (a, b) , pair (6, a)}). 

Definition 3.8. Let t be a term. We define a set of quasi-subterms QSub (t) 
as follows: 



QSub (t) 



{t}, ifteXuA; 

{t}UQSub(ti), if t = priv(ti) ; 

{t} U QSub (ti) U Q Sub (ta) , if t = bin (ti, ta) 

Wuaeciems{L)QSub(p), ift = -(L) 



If T — set of terms, then QSub (T) = IJteT QSub (t). If 5 = {Ei t> ti}-^^ ^ 
is a constraint system, we define QSub (S) = Uteij" i-BiU{ti} QSub (t). 

Example 3. Referring to Example 1, we have 

QSub (t) = {■ ({a, ■ ({6, a, pair (a, b)}) , pair (■ ({6, 6}) , a)}) , 
a, 6, pair (a, 6) , pair (■ ({6, 6}) , a) , ■ ({6, 6})}. 
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Definition 3.9. Let t be a term. We define Vars (t) as set of all the variables 
in t: 

Vars {t) = Xn Sub (t) 

We define Sub(t) as the set of subterms of t and the DAG-size of a term, 
as the number of its different subterms. The DAG-size gives the size of a 
natural representation of a term in the considered ACI theory. 

Definition 3.10. Let t be a term. We define Sub (t) as follows: 

if t = priv (ti) ; 
if t = bin (ti, ^2) 
if t = -{L). 

If T is a set of terms, then Sub (T) = IJter (^)- If = {Ei t> ti}-^^ ^ is 
a constraint system, we define Sub (S) = Utelj" lE^uiu} (^)- 

Example 4. Referring to Example 1, we have 

Sub (t) = {■ ({a, ■ {{b, a, pair (a, b)}) , pair (■ ({6, b}) , a)}) , 
■ ({6, a, pair (a, 6)}) , pair (• {{b, b}) , a) , 
a, b, pair (a, 6) , ■ ({6, &})}. 

Definition 3.11. We define a DAG-size sIzcdag of a term t as sIzcdag (t) = 
I Sub for set of terms T, sizeoAG (^) = I Sub (T)| and for constraint system 
S as sizcDAG (i?) = I Sub {S)\. 

Remark, that for a constraint system such a definition does not polynomi- 
ally approximate a number of bits needed to write it down(cf. Def. 4.1). 

We define a Dolev-Yao deduction system modulo ACI equational theory 
(denoted DY+ACI). It consists of composition rules and decomposition rules, 
depicted in Table 2 where ti,t2, . . . ,tm G T. 

We suppose, hereinafter, that for a constraint system S, QSub {S)r\A 7^ 0. 
Otherwise, we can add one constraint {a} > a to S which will be satisfied by 
any substitution. We denote {priv (t) : t E T} for set of terms T as priv (T). 
We define Vars (iS) = IJiLi Vars (Ei) U Vars (ti). We say that S is normalized, 
iff for all t G QSub (S), t is normalized. 



Sub {t) 



{t} U Sub (ti) , 
{t} U Sub (ti) U Sub (ta) 
IWuaeLSub(p), 
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Composition rules 


Decomposition rules 


ti,t2 '"enc (^1,^2)"' 

^1,^2-^ '"aenc (^1,^2)"' 

ti,t2^ ^pair (^1,^2)^ 

ti, priv {t2) '"sig (ti, priv {t2))~' 

til ■ ■ ■ 1 tm ^ (^1) ■ ■ ■ ) tm)~^ 


enc (ti,t2),^t2^ ^ 
aenc (ti, t2) , '"priv (^2)"' ^ '"^i"' 
pair (^1,^2) 
pair (ti,t2) '"^2"' 
•(^1,...,^^) r^.-i for all i 



Table 2: DY+ACI deduction system rules 



Example 5. We give a sample of general constraint system and its solution 
within DY+ACI deduction system. 

where a,b,c & A and a; G A'. One of the eventual models within DY+ACI is 
a = {x ^ enc (pair (a, 6) , c)}. 

Definition 3.12. Let T = {ii, . . . ,tk} be a non-empty set of terms. Then 
we define 7r(T) as follows: 

n{T)^r.^t,,...,tk)^ 

Remark: n{{t}) = ^t^. 

Definition 3.13. We denote QSub(»S) \ A" as QSub(S', A") or, for shorter 
notation, QSub(5'). 

We introduce a transformation ti{H^''^ (•)) on ground terms that replaces 
recursively all binary root symbols such that they are different from all the 
non-variable quasi-subterms of the constraint system instantiated with its 
model (7, with ACI symbol •. Later, we will show, that 7r(H((j)) is also a 
model of S. 

Definition 3.14. Let us have a constraint system S which is satisfiable with 
model a. Let us fix some a e (^fl QSub (5)). For given S and a we define 
a function H^^" (•) : 7^ ^ 2'^9 as follows: 
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if-^'" (t) = < 



{bin {n{H' 



{a} 
{a} 

{priv (7r(i7'5'- (ti)))} 



if t G (^\QSub(5)); 

if t = a G (^nQSub(5)); 

if t = priv (ti) ; 

if t = bin (ti, 1(2) 

rtn e rQSub(5) 

if t = bin (ti, t2) 

A rtn ^ rQSub(5)(T^ 

if t = ■ (L) . 



if'^''^ (ti) U H^^^ 



it2) 




Henceforward, we will omit parameters and write H (■) instead of if"^'"" (■) 
for shorter notation. 

Definition 3.15. We define the superposition of 7r(-) and H (■) on a set of 

terms T = {ti, . . . , 4} as follows: 7r(i7 (T)) = {tt{H (t)) \ t e T}. 

Definition 3.16. Let 6 = {xi ^-^ ti, . . . ,Xk ^ t^} be a substitution. We 
define tt{H {6)) the substitution {xi 1— > 7i{H (ti)), . . . , 1— ?■ 7r(i7 (^fc))}- 

Note, that dom(7r(i7 (9))) = dom(6'). 

Example 6. We refer to Example 5 and show, that 7t(H (a)) is also a 
model of S. tt{H (enc (pair (a, b) , c))) = 7r(if (pair (a, 6)) U {c}) = vr({a} U 
{6} U {c}) = ■ ({a,6, c}) (we suppose that a -< 6 ^ c). One can see, that 
7r(H (a)) = {x I— 7- ■ ({a, 6, c})} is also a model of 5 within DY+ACI. 

3.1.2. General properties used in proof 

The two following lemmas state simple properties of derivability. 

Lemma 2. Let A,B,C CTg. Then if A C Der (B) and B C Der (C) then 
A C Der (C). 

Lemma 3. Let A,B,C,D C Then if A C Der (5) and C C Der (D) 
then AUC C Der (BUD). 

In Lemma 4 we list some auxiliary properties that will be used in main 
proof. 

Lemma 4. The following statements are true: 



19 



1. For terms t,ti,t2, we have ^■{t,t)^ = ^t^, ^- {ti,t2)^ = (t2,ti)^, 

2. if t and ta are terms, then '~ta^ = ''''ta'^'^ = '"'"t^ a"' = '~t '"a"'"' = 

3. s e QSub (rtn) ^ s = ^s^ 

4. Vs e Sub (rtn) 3^' ^ Sub (t) : s = rg'^ 

5. '"elems (t)"! = elems ('"t"') 

6. r. (rt^n , . . . , rt^n)n = r. (t,, . . . , t„)n; ^^(r) = 7T{rT^) 

7. elems (r- (rt^n , . . . , rt„n)n) = elems (■ (rt^n , . . . , rt^n)) = 
U=l,...,n^ elems (rt,n) 

9. H{t) = H (rtn) 

10. 7r(/7(t)) = n{H{rt^)) = rn{H (t))^ = ^^{H (rt^))^ 

11. 7r(ri U ■ ■ ■ U Tra) = 7r({7r(Ti), . . . , 7r(T^)}) 

12. QSub (QSub (t)) = QSub (t) 

13. QSub(rtn) c rQSub(t)^ 

14. QSub (ta) C QSub (t) a U QSub (Vars (t) a) 

15. Sub (td) = Sub (t) a U Sub (Vars (t) a) 

16. \rT^\ < \T\, \Ta\ < \T\ 

17. elems (t) C QSub (t) C Sub (t) 

18. For term t, sizeDAG ('"i"') ^ sizeDAG (^)/ 

/or set of terms T , sizeoAG ('~^~') ^ sizeDAG (^)/ 
/or constraint system S , sizeDAG {^S^) < sizeDAG [S) 

19. QSub(-({ti,...,tJ)) C {■({ti,...,tJ)}uQSub(ti)---UQSub(tO 

20. Vs G Sub {t) sizeDAG (^^o"^) > sizeDAG {^sa^). 

Proof. We will give proofs of several statements. Some other technical proofs 
are given in Appendix B.l 

Statement 5: This statement is trivial, if t 7^ ■ (L). Otherwise, let t = 

■ {ti, . . . ,tn)- 

• if '"elems (t)"" = {p}, where p 7^ • (Lp). Then '~t~^ = p and then 
elems ('"t"') = elems (p) = {p} = '"elems (t)"'. 

• if '"elems (t)~^ = {pi, . . . ,Pk}, k > 1, where Pi 7^ ■ (Lj) for all i. 
Then '"t"' = -{L), where L ^ {pi, . . . ,pk}. That means, that 
elems (rt^) = Upe{pi,...,Pfe} ^lems (p) = {pi, . . . ,pk}. 
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Statement 6: The first part follows from the definition of normal form and 
Statement 5. The second one directly follows from the first. 

Statement 9: By induction on sizeoAc (t)'- 

• sizeoAG (t) = 1 is possible in the only case: t = a E A and as 
a = '"a"', the equality is trivial. 

• Suppose, that for any t : sizeDAG (t) < k {k > 1), H (t) = H ('"t"') 
holds. 

• Given a term t : sizeoAc it) = k, k > 1. We need to prove that 
H{t)=H{rt-^). 

— ii t = priv(ti), then H (t) = {priv (7r(/7 (ti)))} = (by induc- 
tion supposition) = {priv (vr(i7 ('"ti"')))} = H (priv ('"ti"')) = 
i7(rtn). 

- lit = bin(p,g) and r^n g rQSub(5)cJ^. Then if (^t^) = 
H (bin (rpH , rgn)) = {bin (7r(i/ (^p^)), Ti{H (^g^)))} = (by in- 
duction supposition) = {bin (7r('"if (p)^), 7r(rif (g)"i))} = (by 
Statement 6) = {bin (7r(/7 (p)), 7r(/7 (g)))} = H (bin (p, q)). 

- li t = bin(p,g) and r^n ^ rQgub (5) (T^. Then H {t) = 
H{p)U H (g) = (by induction) = H (r^n) U H (rgn) = (as 
rbin (rpH , rgn)n = rfn ^ rQgub (5) O"^) 

= if (bin (rpH , r^n)) = (r^n) 

— if t = ■ (i^), where L = {ti, . . . ,tm}- Note first, that as 
t = ■ (L), we have for alls G elems (t), sizeoAG (-s) < sizeDAC (^)- 
Then, by Statement 8, H (t) = Upecicms(t) H (p) = (by in- 
duction supposition) = IJpeeiems(t) ('"P"')- other 
part, H{rr) = Upeeiems(rn) ^ (p) = (by Statement 5) = 

Uperclcms(t)n H (p) = Upeelems{t) H i^p^) =H{t). 

Statement 11: From definition of vr and Statement 5, we obtain that 

elems (7r(T,)) = relems(T,)n. Next 7r({7r(Ti), . . . , 7r(T„)}) = r. (L)n 

(here we use (L)^ to capture two cases from definition of normalization 

at once), where L ^ '"elems ({7r(Ti), . . . , 7i(Tm)})~^ = 

^[J^=l,...,m ^elems (T,)^^ = r[j^=i,...,m elems (T,)^, 

while 7r(Ti U ■ ■ ■ U T„) = r. (L')^, where L' ^ ^J.^^^ ^^ elems (T,)^. 

Statement 13: By induction on sizeoAG (t)- 
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• sizeoAG (t) = 1. Then t eAUX. As Q Sub (t) = {t} and t = ^t~^, 
the statement holds. 

• Suppose, that for any t : sizeDAG (t) < k {k > 1), the statement is 
true. 

• Given a term t : sizeoAG (t) = k, k > 1. Let us consider all possible 

cases: 

— t = bin (ti,t2)-0n the one hand, QSub (t) = {t} U QSub (ti) U 
QSub(t2)- On the other hand, '"t"' = bin ('"ti"' , '"^2"') and 
then, QSub (^tn) = {r^} u QSub (^ti^) U QSub (^^2^)- Then, 
as QSub(ri(;in) C rQSub(ti)^, we have that QSub(rtn) C 
rQSub (t)^. 

— t = priv (ti). Proof is similar to one for the case above. 

— t = .{L). We have QSub(t) = {t} U Up6eiems(L) QSub (p). 
From Statement 5 we have elems {''■ (L)^) = '"elems (■ {L))~^, 
and then, QSub (i"- (L)'^) = 

W^} U Upeelems{r.(LP) QSub (p) = 

r{. U Up6eiem.(.(L)) QSub (r^n) c (by supposition) 

C U Upeelems(.(L)) ^QSub = 

m U aeclcms{.(L)) QSub = rQSub 
Statement 14: By induction on sizeoAG (t) 

• sizeoAG (t) = 1. 

— t & A. As ta = t and Vars (t) = 0, the statement becomes 
trivial. 

— t e X. Then QSub (t) a = ta, Vars (t) = {t}; We have 
QSub (ta) C {ta} U QSub (ta). 

• Suppose, that for any t : sizeoAG {t) < k {k > 1), the statement is 
true. 

• Given a term t : sIzcdag (t) = A;, A; > 1. Let us consider all possible 
cases: 

— t = bin(ti,t2)- Then ta = bin (tiO", t2C") and Vars (t) = 
Vars (ti) U Vars (^2)- QSub (ta) = {ta} U QSub (tia) U 
QSub(t2a) C (as sIzcdag (^j) < A;) C {ta} UQSub(ti)aU 
QSub (Vars (ti)a)UQSub (t2)aUQSub (Vars(t2)a) = {ta}U 
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QSub (ti) (T U QSub (ta) o" U QSub ((Vars (ti) U Vars (ta))^-) = 
QSub (t) a U QSub (Vars (t) a). 

— t = priv (ti). Proof is similar to one for the case above. 

— t = ■ ({ti, . . . , tm})- We have ta = ■ ({ticr, . . . , tm<j}) and 
Vars (t) = IJj=i m^^^^('^*)- Then we have QSub(t(T) = 
{^f^} U Upeciems({tia,...,t^.})QSub(p) C (using Statement 17) 
C {ta} U Upeur=i QSub(ti.r) QSub (p) = (as QSub (QSub (p)) = 
QSub (p)) = {to-} U |Ji=i m QSub (ticr) C (as sizeoAG (^i) < 
k) 

^ {MuU=i,...,™(QSub (t,)aUQSub (Vars (t,) (x)) = {tajU 
U=i,...,™QSub(t.)aU QSub ((U=i,...,™Vars(t.)) a) 
= QSub (t) a U Sub (Vars (t) a). 

□ 

Lemma 5. Given a constraint system S and its model a. Then substitution 
7r(H (cr)) is normalized 

Proof. For any x G dom (7r(H (cr))), xn{}i{a)) = 7r{H (xa)) = ^ti{H {xa))^ 
(by Lemma 4). □ 

Lemma 6. For any normalized term t, QSub (t) = Sub {t). 

Proof. By induction on sizeoAG (^)- 

• sizcDAG if) = 1- Then t G X U A, and thus, QSub (t) = Sub (t) = {t}. 

• Suppose, that for any t : sizcDAc (t) < k {k > 1), QSub (t) = Sub (t). 

• Given a term t : sizeoAG (t) = k, k > 1. We need to show that 
QSub (t) = Sub (t). 

- t = bin(ti,t2)- Then QSub (bin (ti, ta)) = {t} U QSub(ti) U 
QSub(t2) = (as sizeoAG (^j) < k) = {t} U Sub (ti) U Sub (ta) = 
Sub (t) 

- t = priv(ti). Then QSub (priv (ti)) = {t} U QSub(ti) = {t} U 
Sub (ti) = Sub (t) 
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— t = ■ (L). As t is normalized, we have that for all p G L, p 7^ 
■{Lp). Then elems (L) ^ L. Thus, we have QSub(t) = {t} U 

Sub (t). 

□ 

In Proposition 1 we remark, that ACI-set of normalized terms has the 
same deductive expressiveness as that set of normalized terms itself. 

Proposition 1. Let T he a set of terms T = {ti, . . . ,tk} ■ Then 7r(T) e 
Der (rT^) and ^T^ C Der ({7r(T)}). 

In Proposition 2 we state that a constraint system and its normal form 
have the same models. In Proposition 3 we show the equivalence, for a 
constraint system, between the existence of a model and the existence of a 
normalized model. As a consequence we will need only to consider normalized 
constraints and models in the sequel. 

Proposition 2. The substitution a is a model of constraint system S if and 
only if (J is a model of^S^. 

Proof. By definition, cr is a model oi S = {E-i > iff Vi G {1, . . . , n}, 

'"ijCr"' G Der (ri^^jcri). But by Lemma 4 we have that '"tjO""' = ^^t.p and 
^Eia'^ = ^^Epa^. Thus, cr is a model of S if and only if a is a model of 
^S^. □ 

Proposition 3. The substitution a is a model of constraint system S if and 
only if^(J^ is a model of S. 

Proof. Proof is similar to one of Proposition 2. □ 

3.2. Ground case of DY+ACI 

In Algorithm 2 we need to check whether a ground substitution a satisfies 
a constraint system S. For this, we have to check the derivability of a ground 
term from a set of ground terms. In this subsection we present such an 
algorithm. 

First, for the ground case we consider an equivalent to DY+ACI deduction 
system DY+ACI' obtained from the first by replacing a set of rules 
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with 

Vs e elems (t) t^^s^, if i = • (L) . 

Now, we show an equivalence of the two deduction systems. 

Lemma 7. t e T)erDY+ACi (E) <(=^ t e T)erDY+Acr (E) 

Proof sketch. We show that every rule of one deduction system can be simu- 
lated by a combination of rules from the other. It is sufficient to show it for 
non common rules. 

The DY+ACr rules Vs G elems (t) t , if t = ■ (L) are modeled 

by successive application of rules Wi ■ {ti, . . . ,tra) '"^i"'- The converse 
simulation of • (ti, . . . , tjn) — >■ ''tp by DY+ACF is based on getting all the 
normalized elements of and, if ji" elems {ti)~^\ > 2 then reconstructing '~tp 
by rule pi, . . . ,pi ^ (pi, . . . ,pi)^, where pi,...,pi are relems {uy. □ 



Algorithm 1: Verifying derivability of term 



Input: A normalized ground constraint E >t 
Output: t e Ber DY+ACi (E) 

Let S := QSub (E) U QSub (t) \ E; 

Let D := E; 



3 while true do 



4 


if exists DY rule I — > r, such 


5 




S 


■ , 




6 




D := 


DU{r}; 


7 


else 






8 




if exists s & S : elems is) 


9 






S:=S\{sy, 


10 






D := DU{s}; 


11 




else 




12 






if exists s E D : elems 


13 








S := S \ elems (s); 


14 








D := D U elems (s); 


15 






else 


16 








return t e D; 


17 






end 


18 




end 




19 


end 






20 end 









C D then 
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Lemma 8. For Algorithm 1 the following statements are true: 

• for any step^, DUS = QSuh{EU {t}) and D f] S = 0; 

• it terminates; 

• for any step, D C BeTDv+Aci {E) . 

The following lemmas will be used to prove correctness of the algorithm. 
Lemma 9. 

• For any decomposition rule I — )■ r of DY+ACF, if I is normalized, then 
r is a quasi-subterm of I. 

• For any composition rule I r of DY+ACF except {ti, . . . ,tm} 

(ti, . . . if ^ is normalized, then I C QSub (r). 

Lemma 10. After the execution of Step 16 of Algorithm 1, if I ^ r is a 
DY+ACF rule, such that I C D and r ^ D, then I ^ r is a composition rule 
andri QSub {EVJ{t]). 

Proof. Suppose, / — )■ r is a decomposition. By Lemma 9 we have that 
r G QSub (/) and thus, r G QSub (D) C D U S. Then r imphes r G S", 
and then. Step 16 must be skipped, as branch 4 or 12 should have been 
visited. 

Thus, / — )■ r is a composition. As algorithm reached Step 16, that means 
r ^ S (otherwise one of three branches must be visited and this step would 
be skipped). As r ^ S and r ^ D, we have r ^ S U D = QSuh {E U {t}). □ 

Lemma 11. Civen a set of normalized terms S such that for any s E S , 
elems (s) C S. Then for any DY+ACF composition rule I — )■ r such that 
I S we have elems (r) C S U {r}. 

Proof. All cases of composition rules except ti, . . . ,tm ^ {ti, . . . , tm)'^ are 
trivial, as for them elems (r) = {r}. For this elems {ti) ^ S for all i, 

then (by Lemma 4, Statement 7) elems (ti, . . . , tm)"') = 
elems (■ (ti, . . . , = IJ^^ elems (ti) C S. □ 

Proposition 4. Algorithm 1 is correct. 

''Consider two sequential assignments as one step 
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Proof. If algorithm returns true, then, by Lemma 8, t E DeiDY+Acr (E). 

Show, that output is correct, if algorithm returns false. Note, that we 
consider values of D and S that they have after finishing the algorithm. 
Suppose that output is false {t ^ D), but t G Der ^ly+Ac/' (E). Then there 
exists minimal by length derivation {Ei}-^^^ ^ where n > 1, D = Eq (as 
D C DeiDY+Acr (E) and t ^ D) and t E En and Ei^i \Ei^ ^ and Ei -^i^^n 
Ei^i for alH = 0, . . . , n — 1. Then, applying Lemma 10 we have Iq — )■ tq is a 
composition, and vq ^ QSub {E U {t}). 

Let m be the smallest index such that there exists s G S* = QSub {E U {t})\ 
D and s G -Em- 
Let k be the minimal integer, such that is a decomposition. 
Show, k < m. Suppose the opposite, then s is built by a chain of 
composition rules from D. If Im-i — ^ fm-i (where Tm-i = s) is 

• a rule in form of {ti, . . . , tc} — > '"■ (^i, • • • , ^c)^, then elems (s) 7^ {s} 
(otherwise it contradicts to minimality of the derivation) and from 
Lemma 11, elems (s) C E^n-i {itl 7^ 1, otherwise this step would be 
executed in the algorithm). As s G S*, then elems (s) C QSub(s) C 
QSub {E U {t}). If elems (s) C D then we got contradiction with with 
the fact, that this step would be executed in the algorithm. If there 
exists e G elems (s) and e ^ D (that means, e E S), then we get a 
contradiction with the minimality of m, as e G S" was deduced before. 

• any other composition rule, then by Lemma 9, /^-i ^ QSub (s), and 
thus, Im-i DU S. Similarly to the previous case, m 7^ 1 and we get a 
contradiction with either minimality of m, or with the fact, that the 
algorithm would have to add s into D. 

Note, that this also shows, that decomposition rule is present in derivation. 

Show, Ik ^ D. Suppose the opposite. Then by Lemma 9, we have ru ^ D 
what contradicts to -Efc+i \Ek ^ ^- Thus, at least one element from is not 
from D. Let us consider all possible decomposition rules Ik r^: 

• {pair (ti, t2)} — ^ '~ti'^- We know, that pair(ti,t2) is not in D, thus, 
it was built by composition. As Ei are normalized, the only possi- 
ble way to build by composition pair(ti,t2) from normalized terms 
is {^1,^2} — ^ pair(ti,t2) (other ways, like pair (ti, ^2) , pair (ti, ^2) 

I"- ({pair (^1,^2) 5 pair (ti, ^2)})"' would contradict the minimality of the 
derivation). Thus, ti was derived before (or was in D), i.e. ti G E^. 
That contradicts to Ek+i \ -Efc 7^ 0- 
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• {pair (ti, t2)} — ^ '"^2"'- Similar case. 

• {enc (ti, t2) , '"^2"'} — > '"^1^- The case where enc (ti, ^2) ^ D has similar 
explanations as two cases above. Thus, enc(ti,t2) £ D. That means, 
t2 G QSub(i?U{t}) and ^2 ^ -D, i.e. t2 G S. This means, ^2 was 
derived before and t2 E S, what contradicts to k < m. 

• {aenc (^1,^2) , '"priv (^2)"'} ^ '"^i"' is a similar case to previous one. Note, 
that if priv (12) is not in D, that it must be obtained by decomposition. 

• t ^ '"s"', where s G elems (t) and t = ■ (L). By Lemma 11, elems (if:) C 
Ek, that contradicts minimality of derivation {Ek+i \ -E^ 7^ 0). 

□ 

3.3. Existence of conservative solutions 

In this subsection we will show that for any satisfiable constraint system, 
there exist a model in special form (so called conservative solution). Roughly 
speaking, a model in this form can be defined per each variable by set of 
quasi-subterms of the constraint system and set of atoms (also from the 
constraint system) that must be prived. This will bound a search space for 
the model (see § 3.4). 

First, we show, that on quasi-subterms of constraint system instantiated 
with its model, the transformation 7r(i7 (■)) will be a homomorphism modulo 
normalization. 

Proposition 5. Given a normalized constraint system S and its normalized 
model a. For all t E QSub(>S), rtvr(H(o-))^ = ^7r{H (ta))-^. 

Proof. We will prove it by induction on |Sub {t)\, where t is normalized. 

• Let I Sub (t) I = 1. Then: 

— either t E A. In this case t E {An QSub (5)), and as t/i = t for 
any substitution /i, then n{H (ta)) = ^{H (t)) = n{{t}) = t and 
1 7r(H (a)) = t. Thus, 1 7r(H (a)) = tt{H (ta)). 

— OT t E X. As cr is a model and t E QSub (5), we have t E dom (a), 
and, by definition, t E dom (7r(II (a))). Then, by definition of 
7r(H (a)), tn(R (a)) = n{H (ta)). 
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• Assume that for some A; > 1 if |Sub(t)| < k, then '"t 7r(H (a))"' = 

• Show, that for any t such that |Sub {t)\ > A; + 1, where t = bin {p, q) 
or t = priv (g) or t = ■ (ti, . . . ,tm), but | Sub {p)\ < k, \ Sub (g) | < k 
and |Sub(tj)| < k, for aU i G {l,...,m}, statement ''tniB. = 
'~tt{H {ta))~^ is stiU true. We have: 

— either t = bin {p, q). Ast = bin (p, q) G QSub (S) =^ p E QSub {S) 
and q G QSub (S). As |Sub {p)\ < |Sub and from the induction 
assumption, we have '~p7r(H (a))"' = ^-k^H (pcr))"^. The same holds 
for q. 

Again, since bin (p, q) a E QSub (S) a (as bin (p, g) ^ A" and t G 

QSub (5)) we have that ^7i{H (bin (p, g) = 

^tt{H (bin (per, ga)))^ = ^n{H (^bin (per, ger)^))^ = 

rnln (bin (^pa^ , rga^)))^ = 

r7r({bin (7r(i/ (rpa^)), 7r(/J (rg(T^)))})^ = 

r7r({bin (r7r(/7 (pa))^ , rTr{H (g(T))^)})^ = 

rbin {'~tt{H {pcr))~^ , r7r(i^ (go"))^)^ = 

rbin (rp7r(H (ct))^ , rg7r(H ((t))^)^ = 

rbin (p7r(H (a)),g7r(H (a)))^ = 

rbin (p, g) 7r(H (a))^ = 7r(H ((t))^. 

— or t = ■ (ti, . . . ,tm)- As t is normahzed, it imphes that for all 
i G {1, . . . , m}, ti are not in form of ■ (Lj) and then ti G QSub (5), 
and thus, we have ti G QSub (5) A ^n^H {tia)p = rt- 7r(H (cr))^. 
7r(i/(ta)) = 7r(i7(-(tia,...,t„a))) = 7r(/7(tia)U---Ui7(t^a)) = 
(by Statement 11 of Lemma 4) 

= 7r({7r(ff(tia)),...,7r(i7(t™a))}) = 
7r({rtivr(H(a))n,...,rt^7r(H(cr))n}) = 
r.(rti7r(H((T))^...,rt^7r(H (a))n)n = 
r.(ti7r(H (a)),...,t„7r(H (a)))^ = 
r(.(ti,...,t„))7r(H (a))n = ri^(H(a))n 

— or t = priv (g). Then g G QSub (5). 

7r{H (ta)) = 7r({priv (7r(i7 (ga)))}) = ^priv (7r(i/ (ga)))^ = 
rpriv (g 7r(H (a)))^ = ^priv (g) 7r(H (a))^ = ^t 7r(H (a))^. 

Thus, the proposition is proven. 

□ 
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Now we show, that relation of derivabiUty between a term and a set of 
terms is stable with regard to transformation tt{H (■)). 

Lemma 12. Given a normalized constraint system S and its normalized 
model a. For any DY+ACI rule h, . . . ,lk f, 
niH (r)) G Der {{niH (h)), . . . , 7r(i7 (h))}). 

Proof idea. We proceed by considering all possible deduction rules. To give 
an idea, we show a proof for only one rule (see full proof in Appendix B.3): 
aenc (ti, ^2) , '"priv (^2)"' — ^ '"^i"'- Here we have to show that 7r(if ('"ti"')) is 
derivable from {vr(if (aenc (ti, ^2))), 7r(if (^priv (t2)~'))}- Consider two cases: 

• 3m G QSub (S) such that '"aenc (^1,^2)"' = '"mct"'. Then 
7r{H (aenc (^1,^2))) = aenc {7r{H (ti)), 7r{H (^2))), 

and then 7r{H {^ti^)) = 7r{H (ti)) G 
Der ({aenc {7r{H (ti)), niH (^2))) , ^priv (7r(i/ (ta)))^})- 
On the other hand, tt{H (^priv (^2)"')) = vr(iJ (priv (t2))) = 
7r({priv {7r{H (t2)))}) = ^priv {it{H (t^)))^. 

• $u E QSub (5) such that ^aenc (^1,^2)"' = '~ua~'. Then 

7r{H (aenc (^1,^2))) = T^iH (ti) U H (^2))- Using Proposition 1, we have 
{ti)UH {ti)^ C Der ({7r(i/ (aenc (ti,t2)))}), thus (by Lemma 4) 
^H lti)^ C Der ({7r(iJ (aenc (ti,t2)))})- And then, by Proposition 1 
we have that 7t{H (ti)) G Der ('"if (ti)^). Therefore, by Lemma 2, 
n{H (rt^n)) = Tr{H (t,)) G Der {n{H (aenc (^1,^2))))- 

□ 

Using Proposition 5 and Lemma 12 we will show, that transformation 
7r(iJ (■)) preserves the property of substitution to be a model. 

Theorem 1. Given a normalized constraint system S and its normalized 
model a. Then substitution 7r(H (cr)) also satisfies S. 

Proof. Suppose, without loss of generality, S = {Ei t> ti}^^^ ^. Let us take 
any constraint {E \> t) G S. As o" is a model of S, there exists a derivation 
D = {Aq, . . . , Ak} such that Aq = ^Ea^ and ^ta'^ G Ak. 

By Lemma 12 and Lemma 3 we can easily prove that if /c > 0, tt{H {Aj)) C 
Der (7r(if (y4j_i))) , j = l,...,k. Then, applying transitivity of Der (■) 
(Lemma 2) k times, we have that n{H {Ak)) C Der (7r{H {Aq))). In the 
case where k = 0, the statement tt{H {Ak)) C Der {^{H {Aq))) is also true. 
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Using Proposition 5 we get tt{H (Aq)) = tt{H (Ea)) = 'k{E {(r))~^, as 
E C QSub(5). The same for t: tt{H (ta)) = 7r(H (cr))^, and as ^ta^ G 
Ak, we have ^tn{R{(T))^ G 71(11 {Ak)). Thus, we have that ^t7r{R{a))^ e 
TTiHiAk)) C Der(7r(if(Ao))) = Der (r^ 7r(H (a))^), that means 7r(H((T)) 
satisfies any constraint of S. 

□ 

From now till the end of subsection we will study a very useful property of 
7r(H (cr)). Proposition 6 and its corollary show, that if constraint system has a 
normalized model (a) which sends different variables to different values, then 
there exists another normalized model (vr(H (cr))) that sends any variable of 
its domain to an ACI-set of some non- variable quasi-subterms of constraint 
system instantiated by itself and some private keys built with atoms of the 
constraint system. 

Lemma 13. If ^ua^ = enc(p, g), cr is normalized, u = '~u~^, u ^ X and 
xa 7^ ycr, X 7^ y, then there exists s G QSub('u) such that s = enc (p', g') 
and Tscr"! = enc [p, q). The similar is true in the case of ^ua^ = pair (p, q), 
^ua^ = aenc [p, q), ^ua~^ = sig [p, q) and for ^ua^ = priv (p). 

Proof. As u = '"m"' and '~ua~^ = enc [p, q), we have: 

• u not in form of ■ (L). Then, as u ^ X and '~ua~^ = enc [p, q), we have 
u = enc [p', q') (where '~p'(T~^ = p and '~q'a~^ = q). Then we can choose 
s = enc {p', q') = u E QSub (u). 

• u = ■ (ti, . . . ,tm), m > 1, as M = '"■u"!. Then, for all i, ti is either a 
variable, or enc (p^, q[). But, as xa ^ ya, x ^ y and as a is normalized, 
we can claim, that {ti, . . . contains at most one variable. Then, 
as m > 1, there exists i such that tj = enc (p'j, g^). Then by definition 
of normalization function, and from ^ua^ = enc [p, q) we have, that 
•"elems (ucr)~i = {enc(p, g)} and as tia is an element of ua, we have 
•"enc (p'j, q'j) a~^ = enc (p, g). Thus, we can choose s = tj, as G QSub (u) 
and ti = enc (p-,g'). 

The other cases (pair, priv, etc..) can be proved similarly. 

□ 

Proposition 6. Given a normalized constraint system S and its normalized 
model a such that for all x,y E dom(cr), x ^ y =^ xa ^ ya. Then 
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for all X G dom (7r(H (a))) there exist G N and Si, . . . , G QSub {S) U 
priv (QSub (5) fl A) such that root (sj) 7^ ■ anc? 
a;7r(H (a)) = 7r({si 7r(H (a)), . . . , s,. 7r(H (a))}). 

Proof. By definition, a:7r(H (a)) = 7r(i/ (xcr)). Let us take any s & H {xa) 
(note, that s is a ground term). Then, by definition of H (■) we have: 

• either s E A. Then, by definition of if (■), s G (^ fl QSub (5)). Thus, 
S7r(H (a)) = s, s G QSub(^), s ^ ■ (L); 

• or s = bin {j{H (ti)), ■n{H (^2))) and there exists u G QSub {S) such 
that r-ucr^ = rbin (^1,^2)"' = bin ('"ti"' , '~t2~')- As all conditions of 
Lemma 13 are satisfied, then there exists v G QSub {u) such that 
^va^ = bin ('"tp , '"^2"') and v = bin (p, g) and as m G Q Sub (5*) then 
V G QSub (S). By Proposition 5, 

rv 7r(H (a))^ = n{H (va)) = 7i{H (rya^)) = 7i{H (bin (ti, t2))) = 
7r({bin {7r{H (ti)), 7r(iJ (t2)))}) = bin (7r(i7 (ti)), 7r(iJ (^2))) = s. That 
means that there exists v G QSub (S) such that v 7^ ■ (L) and 
s = rt>7r(H (o-))^. 

• or s = priv {n{H (ti))). In this case, as s is ground, tt{H (ti)) must be 
an atom, moreover, by definition of H {■), this atom is from 

{A n QSub (5)). Therefore, s = priv (a), where a E An QSub (5) (and 
of course, s 7^ • (L)). 

Thus, for all s G -ff (xcr), there exists 

V G (QSub (5)) U priv (QSub (5) n ^) \ A" | s = ^t; 7r(H (a))^. Therefore, as 
a: 7r(H (a)) = 7r(iJ (xcr)), we have that x 7r(H (cr)) = 

7r({rsi 7r(H (a))^ , . . . , r^, vr(H (cr))^}) = 7r({si 7r(H (a)), . . . , 7r(H (a))}), 

where Si, . . . ,Sk G QSub (5) U priv (QSub (5) fl ^) and 

Si 7^ ■ (L) ,V1 < z < A;. That proves the proposition. □ 

Corollary 1. Given normalized constraint system S and a' — its normalized 
model, such that x ^ y =^ xa' 7^ ya' . Then there exists a normalized 
model (J of S such that for all x G dom (cr) there exist G N and si, . . . ,Sk G 
QSub (S')Upriv (QSub (S) fl A) such thatxa = 7r({sicr, . . . , Skcr}) and Si 7^ sj, 
ifi 7^ j; Si ^ ■ (L) ,Vz. 

Any normalized model with property shown in Corollary 1 we will call 
conservative. 
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3.4- Bounds on conservative solutions 

To get a decidability result, we first show an upper bound on size of 
conservative model and then, by reducing any satisfiable constraint system to 
one that have conservative model and showing that reduced one is smaller 
(by size) than original one, we obtain an existence of a model with bounded 
size for any satisfiable constraint system. 

Lemma 14. Given a normalized constraint system S and its conservative 
model a. Then for all x e Vars (5) we have QSub (xcr) C rQgub (^S) U 
priv(QSub {S)r}A). 

Proof. Given a ground substitution a, let us define a strict total order on 
variables: x \Z y <^==^ (sizeoAG i^a) < sizeoAG (?/cr)) V (sizeoAG (xa) = 
sizeDAG (ycr) Ax -<y). 

By Proposition 6 for all x xa = n{{sfa, . . . , sl^a}), where G (QSub {S)\ 
X) U priv (QSub [S) n A) and ^-{L). 

Let us show that if ?/ G Vars (sf ) for some i, then y \Z x. Suppose, that 
y G Vars (sf ) and x \Z y. Then sIzcdag (xcr) = sIzcdag (7r({sfcr, . . . , sl^a})) = 
sizeDAG (^- (sf a, . . . , sl^^a)^) > (by Lemma 4) > sIzcdag (^sfa^) > 
sizcDAG ('"?/c""^)) because we know that s^ = bin (p, g) or s^ = priv (p) and 
y G Vars (sf ) (for example, in first case, sIzcdag ('"■^^o""') = 
sizeoAG (bin {''pa'^ , ^qa^)) = 1 + sIzcdag ({^P^-"^ , ^Q'^'^}) and 
since y G Vars {{p, q}), using Statement 20 of Lemma 4, we get 

sizeDAG (^sfc^^) > 1 + sizcDAG {'~ycr^)) And as sIzcdag {'~ycr^) = sIzcdag {ycr) 
That means, y \Z x. Contradiction. 

Now we show by induction main property of this lemma. 

• let X = minc(Vars (5)). 

Then xa = 7r({sjcr, . . . , sl^^a}) = i"- (sfcr, . . . , s^^cr)"' and all sf 
are ground (as there does not exists y \Z x). Then xa = {sf, . . . , slx)~^. 
We have that QSub (xa) = {r- (s^, . . . , s^,)^} U QSub (sf) U ■ ■ ■ U 
QSub(s^.) C rQSub(5)a^ U priv(^nQSub(5)), as for any s G 
QSub(sf), s E Tg and s G QSub (5) or s = priv (a) or s = a, 
where a G QSub (S) fl A, therefore s = '~s~^ = sa E ^QSub (iS) a~^ U 
priv (QSub (S) n A) and r. (s^, . . . , s^^)n = xa E rQSub (S) a^. 

• Suppose, that for all z \Z y we have 

QSub (za) C rQSub (Sa)^ U priv (QSub (S) n A). 
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• Show, that QSub {ya) C QSub {Sa) U priv (QSub (5) n ^). We know 
that ya = Ti{{s\a, . . . , s\ya}) = i"- (sfcx, . . . , s^^a)"' and for any z G 
Vars (sf), z \Z y. Then we have QSub (ya) = 

{ya} U QSub(rs5'cr^) U ■■■ U QSub (rs^^a^). We know that ya e 

rQSub(5)o-^. Let us show that QSub(rsfcr^) C rQSub(5)cr^ U 

priv (QSub (S) HA). By Lemma 4 we have QSub (rs^cr~i) C 

rQSub (sfa)^ C rQSub (sf) a U QSub (Vars (sf) a)^ = 

rQSub (sf) a^ U QSub (Vars (sf) a). We can see that rQSub (sf) a^ C 

rQSub (5) U priv (QSub (S) n ^) (as e QSub (5) U 

priv (QSub (5) H ^)); and by induction supposition and by statement 

proved above we have QSub (Vars (sf) a) C rQSub (5) a~^ U 

priv (QSub (5) n^). 

Thus, QSub (ya) C rQSub (S) a^ U priv (QSub (S) n ^). 

□ 

Proposition 7. For normalized constraint system S that have conservative 
model a, for any x G Vars (S) we have sizeoAG i^a) < 2 x sizeoAG (S). 

Proof. As |rSub (5) cr~'| < |Sub(5)cr| < |Sub(iS)| = sizeDAG('5), we have 
(using the fact that a is normahzed and Lemma 14) that |Sub(x(T)| = 
|QSub(xa)| < |rQSub(5)(T^ U priv(^nQSub(5))| < |rQSub (5) + 
|priv(^n QSub(5))| < sizeoAc (5) + |^ n QSub(5)| < 2 x sizeDAG('5); 
thus, sizeoAG (xa) < 2 x sizeoAG {^)- n 

From this proposition and Corollary 1 we obtain an existence of bounded 
model for a normalized constraint system that have a model sending different 
variables to different values. We will reduce an arbitrary constraint system to 
already studied case. The target properties are stated in Proposition 8 and 
Corollary 2. 

Lemma 15. Given any constraint system S and any substitution 9 such 
that dom (6^) = Vars (5) and dom (6') 6* C dom(6'). Then sIzcdag ('5^) < 
sizeDAG {S). 

Proof. From Lemma 4 we obtain size^AG {SO) = |Sub {S9)\ = |Sub (5) 9 U 
Sub (Vars (5)^) I, but Vars (5) ^ C dom (^) = Vars (5) (Vars (cSa) consists 
only of variables), and then Sub (Vars (5) 6') = Vars (5) 6*. As Vars (5) C 
Sub (S), we have Sub (S) 9 U Sub (Vars (S) 9) = Sub (S) 9. 
Thus, sizcDAG ('56') = |Sub (5) 6^1 < |Sub (5)| = sIzcdag (S). □ 
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Definition 3.17. Let a and b be substitutions. Tlien a\5\ is a substitution 

sucli tliat dom (o"[(5]) = dom [S) and for all x G dom (cr[5]), xo\S\ = {x6)(7. 

Lemma 16. Let 9 and o he substitutions such that dom (^) = dom (a), 
dom (o") C dom (6) and a is ground. Then, for any term t, (t9)a = ta[6]. 

Proof. When apply 6 to t, every variable x of t such that x G dom (6) is 
replaced by x6; then we apply a to t6: every variable y of t6 is replaced by ycr, 
thus, every variable x from dom (6) will be replaced to {x6)cr (as dom (6) 9 = 
dom((j)); and no other variables will be replaced (as dom (cr) C dom(^)). 
Thus, we can see that it is the same as in definition of cr[6]. □ 

Proposition 8. Given any satisfiable constraint system S. Then there exists a 
model cr of S such that for any x G dom [a), sIzcdag (^^o") < 2 x sIzcdag (^'5"^) 

Proof idea. Given a normalized model a' of S we build a substitution 6 that 
maps different variables whose a'-instnatces are the same to one. In this way 
we obtain a new constraint system and its normalized model on which we 
can apply Corollary 1 and get its conservative model a", and by applying 
Proposition 7 we get a bound on size for this model. On the other part, we 
use Lemma 16 to show that ct"[6] is a model of '~S~^. And then, using obtained 
bound and Lemma 15 show existence of a model with stated property. The 
detailed proof is given in Appendix B.4 

□ 

Corollary 2. Constraint system S is satisfiable if and only if there exists a 
normalized model of S defined on Vars (S) which maps a variable to a ground 
term in T{A fl QSub ('"5"') , 0) with size not greater than double sIzcdag {S). 

Using this result, we propose an algorithm of satisfiability of constraint 
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system (Algorithm 2). 
Algorithm 2: Solving constraint system 
Input: A constraint system S = {Ei > „ 
Output: Model cr, if exists; otherwise _L 

1 Guess for every variable of 5 a value of ground normalized 
substitution a with size not greater than 2 x sIzcdag (S); 

2 if a satisfies Ei > tj for all i = 1, . . . ,n then 

3 I return a 

4 else 

5 I return _L 

6 end 



Proposition 9. Algorithm 2 is correct. 

Proof. Let a be an output of Algorithm 2. Then cr is a ground substitution 
and a satisfies all constraints from S' and therefore, satisfies all constraints 
from S . This means, cr is a model of S. □ 

Proposition 10. Algorithm 2 is complete. 

Proof. Suppose, S is satisfiable. Then, by Corollary 2, there exists a guess of 
value of ground substitution on every element of Vars {S) with size not greater 
than 2 x sIzcdag (S) which represents a model cr of S. Thus, algorithm 2 will 
return this cr. 

□ 

4. Complexity analysis 

In this section we present complexity classes of proposed algorithms. First, 
we expose what we use as a representation of constraint systems to justify the 
selected measure of algorithms inputs. Then, we notice that normalization 
algorithm is polynomial in time. After that we will show the polynomial 
complexity of the ground derivability algorithm. And as a consequence of 
the results given before, we obtain that the proposed algorithm for solving 
general constraint system within DY+ACl model is in NP. 

To reason about complexity, we have to define a size of its input. For 
terms and set of terms, we will use sIzcdag (■) + IE (■)!, where E (■) is a set 
of edges of DAG-representation of its argument. For system of constraints 
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S = {Ei t> ti}^^^ ^ we will use n x sizeoAG {<S)+ |IE {S)\ . The justification 
is given below. 

Definition 4.1. D AG-representation of a constraint system S = 
{Ei t> n is a tagged graph with labeled edges G = (V, E,tag) (V is a 

set of vertices and E is a set of edges; tag is a tagging function defined on V) 
such that: 

• there exists a bijection / : V t-)- Sub (5); 

• G V tag (f ) = (s, m), where 

- s = root ifiv)); 

— m is 2n-bit integer, where m[2i — 1] = 1 <^=^ f{v) G Ei and 
m[2t] = 1 ^ f{v) = ti. 

• Vi^V2 eE ^ 3peT : (3bin : f{vi) = hm{f{v2),p)) V f{vi) = 
priv(/(t;2)); 

• f 1 — )■ f 2 G E 3p eT : 3 bin : /(t'l) = bin (p, /(f2)); 
. t;i 4 G E ^ /K) = ■ (L) A = /(t;2); 

Example 7. A constraint system 

{enc (a, x) , pair (6, enc (a, a)) , c} >a 
S = { {priv (6) , c} >y 

{enc (sig (a, priv (c)) , , aenc (x, 6)} > pair (enc (a, x) , c) 

will be represented as shown^ in Figure 3. Nodes of this graph represent an 
element from Sub (5) by indicating its root symbol (first part of its tag) and 
pointers to the children. 

Remark that this representation can be refined, as we know that RHS of 
a constraint is exactly one term. That is why we could tag a node not with 
2n bits but with n + [log(n + 1)] bits (concerning the second component of 
the tagging function). 



'''Label "1" (resp."2") of an edge is represented by a left (resp. right) side of its source 
node 
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00 00 00 




00 10 00 10 00 00 00 00 00 



Figure 3: D AG-representation of constraint system S 

The shown representation can be written in not more than P{n x |V(4S)| + 
|E {S)\) bits of space, where V(-) is a set of edges in the DAG-representation, 
and P is some polynomial with non- negative coefficients. As we have a 
bijection between V(«S) and Sub (5), we obtain |V(<S)| = sIzcdag On the 
other hand, as we are not interested in rigorous estimation of complexity, 
but work in a polynomial class, we will estimate complexity of algorithms by 
taking n x sIzcdag i'S) + |IE {S)\ as the measure of constraint system S. 

The DAG-representation of a term t has the similar structure as it was 
shown for constraint system except that it does not need the second part of a 
tagging function: we need only root {f{v)) as a node's tag. The size of this 
representation will be polynomially bounded by sIzcdag {t) + Thus 
we give the following definition: 

Definition 4.2. The measure of term t is defined as: measure (t) = sizcoAG (^)+ 
|E {t)\. For a constraint system 5 = {E^ > ti}i=i its measure: measure (S) — 
n X sizcDAG {<S) + |IE(5)|. 

Note that for the normalized terms and constraint systems, number of 
edges in their DAG-representation are polynomially limited w.r.t. the number 
of vertices: 
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Lemma 17. For any normalized term t, |IE(t)| < (sizeoAG (^))^- ^or any 
normalized constraint system S, \E{S)\ < (sizeDAG {<S)Y- 

Proof. Since the term (resp. constraint system) is normalized, we cannot 
have more than two edge between two nodes. It evidently holds for binary 
and unary nodes; for -nodes it holds because of normalization: if a -node 
has two edges to one child, the term is not normalized (one of these edges 
should have been removed). Therefore, as the graph is directed and acyclic, 
with as maximum two edges between two nodes, we have not more than 
sizeoAG (x) X (sizeDAG (x) — 1) edges (where x is a term t or constraint system 
S). □ 

4-1- Satisfiability of a general DY+ACI constraint systems is in NP 
Lemma 18. Given a term t. Normalization can be done in polynomial time 
on measure {t). The same holds for a constraint system S: normalization can 
be done in polynomial time on measure (S). 

Proof idea (for the case of terms). The algorithm of term normalization works 
bottom-up by flattening nested ACI-sets, sorting children of ACI-set nodes, 
merging duplicated nodes while removing unnecessary duplicating edges and 
removing nodes without incoming edges (except the root-node of t). □ 

Proposition 11. The general constraint system within DY+ACI satisfiability 
problem, that Algorithm 2 solves, is in NP. 

Proof. Algorithm 2 returns a proof for the decision problem if it exists. We 
have to show, that the verification of this proof takes a polynomial time with 
regard to the input problem measure. To do this, we will normalize Sa and 
then apply algorithm of checking ground derivability. Using the fact that 
sizeDAG {x<^) < 2 X sizeDAG {<S) and polynomial complexity of the normal- 
ization and the ground derivability, we can overapproximate the execution 
time with polynomial on measure (S). The details of the proof are given in 
Appendix B.2. □ 

On the other hand, we can reuse a technique presented in [14] to show 
that the satisfiability of a constraint system is an NP-hard problem. The 
authors encoded 3- SAT problem into an insecurity problem of a single-session 
sequential protocol. Because the steps of the protocol are linearly ordered, 
the finding of an attack is reduced to the satisfiability problem of a single 
constraint system. 
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Theorem 2. Satisfiability of general DY+ACI constraint systems is NP- 
complete. 

4-2. Ground derivability in DY+ACI is in P 

Proposition 12. Algorithm 1 has a polynomial complexity on 

SizCDAG {EU{t}). 

Proof. We will give a very coarse estimate. 

First remark, that in any step of algorithm, IS"! and \D\ don't exceed 
IQSub {EU{t})\. 

Building QSub(i?) U QSub(t) takes linear time on sizeoAG (-E U {t}). 
Building S will take not more than O {\E\ x |QSub {E) U QSub that is, 
not more than O ((sIzcdag {E U {t}))^). 

The main loop has at most |QSub [E U {t})\ — \E\ steps. Searching for 
DY rule with left-hand side in D and right-hand side in S is not greater that 
0{\S\ X \D\^) and thus, not greater that 0((sizeDAG {E{J{t])f). The next 
if can be performed in 0{\S\ x \D\ x (sIzcdag {E U {i}))) steps and the last 
if can be also done for cubic time. The check done in return statement is 
linear. And finally, thanks to the Statement 17 of Lemma 4, we can easily 
justify the claimed complexity. □ 

5. Satisfiability of general DY constraint system 

The previous result on constraint solving for DY+ACI theory can be 
projected to the classical DY case. We cannot apply it directly, as in the 
resulting model we will probably have an ACI symbol. Thus, we need to 
prove the decidability of DY case. The scheme we follow to solve a constraint 
system within DY deduction system is shown in Figure 4. 

First, we can show that if a constraint system is satisfiable within DY, 
then it is satisfiable within DY+ACI (Proposition 13). 

Second, as we know, we can find a model of a given constraint system 
within DY+ACI. 

Third, we will transform the model obtained from previous step (which is 
in DY+ACI) in such a way, that the resulting substitution will be a model 
of initial constraint system within DY(Theorem 3). The idea of satisfactory 
transformation 5 is simple: we replace any ACI list of terms with nested 
pairs: ■ ({ti, . . . , t„}) we replace with pair (ti, pair (..., t„)). Note, that this 
transformation will have a linear complexity and the transformed model will 
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Figure 4: Proof Plan 



have the DAG-size not more than twice bigger than initial. This gives us a 
class of complexity, which is NP, for the problem of satisfiability of general 
constraint system within DY model. 

Definition 5.1. We define a replacement 6 (t) : 7^ h-> 7^ in the following 

way: 

t, ate XUA; 
bin(5(p) ,(5(g)) , if t = bin (p, g) , 
5 (t) = •{ priv (5 (p)) , if t = priv (p) ; 

5{h), ift = -(ti); 
^ pair {5 (ti) , 5 (■ (ts, • • • , tm))) , if t = ■ (ti, ...,tm),m>l; 



Definition 5.2. Given substitution a. Then 5 (cr) = {s — )■ ^ (2;cr)}^gdom((7)- 
For TCrg,5{T) = {5 (t) : t e T}. 

Let us recall classical Dolev-Yao deduction system (DY) in Table 3. 



Composition rules 


Decomposition rules 


ti,t2 enc (^1,^2) 

ti,t2 aenc (^1,^2) 

ti,t2 pair (^1,^2) 

ti, priv (t2) sig (ti, priv (ta)) 


enc (^1,^2) ,h ^ ti 
aenc (ti, t2) , priv (^2) ii 
pair (ti,t2) ^1 
pair (ti,t2) ^2 



Table 3: DY deduction system rules 
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Definition 5.3. A constraint system S standard, if for all s e Sub(<S) 

root (s) 7^ ■. The definition is extended in natural way to terms, sets of terms 
and substitutions. 

We can redefine the notion of derivation for Dolev-Yao deduction system 
in a natural way, and denote it as DeroY- 

Lemma 19. Any standard constraint system is normalized. 

Lemma 20. Let t be a standard term, a be a normalized substitution. Then 
ta is normalized. 

Proposition 13. // a standard constraint system S has a model a within 
DY deduction system, then S has a model within DY+ACI deduction system. 

Proof. It is enough to consider the same model a in DY+ACI. As Sa is 
normalized and as DY+ACI includes all the rules from DY, it is easy to show 
using the same derivation that proves cr to be a model in DY, that a stays a 
model of S in DY+ACI. □ 

The goal of the following reasoning is to show that we can build a model 
of a constraint system within DY from a model of this constraint system 
within DY+ACI. 

Lemma 21. For any DY+ACI rule li, . . . ,lk -+ r, if li are normalized for 
all i = 1, . . . , k then 6 (r) G Der/^y {{6 (/i) , ■ ■ ■ ,S {Ik)})- 

Proof. Let us consider all possible rules: 

• ^1,^2^ '"pair (ii,i2)"' 

As ti and t2 are normalized, then i"pair {ti, t2)~^ — pair {ti, t^)- We can 
see, that 5 (pair (ti, ta)) = pair {5 {ti) , 5 (ta)) e Der^iy {{5 {ti) , 5 (ta)})- 

• ^1,^2 — > '"enc (ti, ta)"'- Proof of this case can be done by analogy of 
previous one. 

• ^1,^2 '"aenc (ti, ta)"'. Proof of this case can be done by analogy of 
previous one. 
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ti, priv (ta) ^sig (ti, priv (t2))^- 

As ti and priv(t2) are normalized, then ^sig (t^, priv (t2))~' = 
sig (ti, priv (^2))- We can see, that 6 (sig (ti, priv (^2))) = 
sig {S ih) , 5 (priv (t2))) = sig (S (ti) , priv {d (^2))) G 
BeiDY {{6 (ti) , priv {5 (^2))}), but priv (5 (^2)) = 5 (priv (^2)). 

The fact, that elems (ti, . . . , tm)~^) = Uj=i m elems (tj) follows from 
ti = ^tf^ (for all i) and Lemma 4. 

We can (DY)-derive from {5 {ti}} any term in 5 (elems (tj)), trivially, if 

ti 7^ ■ (L) and by applying rules pair (si, S2) Si and pair (si, S2) 
S2 otherwise (proof by induction on size of ti). 

One can observe, that 5 {t) is a pairing (composition of pair (-, ■) operator 
with itself) of 5 (elems (t)) (by definition of 6 {■) and normalization 
function). And then, as 6 (t) is limited in size, we can (DY)-derive 

6 (t) from 6 (elems (t)) by iterative use of rule Si, S2 — > pair (si, S2), if 
needed. 

Thus, first we can derive 6 (elems (tj)) for all i, and then rebuild (derive 
with composition rules) 6 (ti, . . . ,tm)~')- 

enc (ti,t2),rt2^ ^ ^tr. 

As enc(ti,t2) is normalized, then ti = '"ti"' and ^2 = '~i^2~'- Thus, 
6 (ti) G Der£)y ({enc (5 (ti) , 6 (t2)) , S (^2)}) and this is what we need, as 
5 (enc (ti,t2)) =enc (5 (ti) , 5 (t2)). 

pair (^1,^2) '~ti~^- Similar case. 

pair (^1,^2) '"^2"'- Similar case. 

aenc (^1,^2) , '"priv (^2)"' — > '"^i"'- Similar case. Note, that S (priv (t2)) = 
priv (5 (t2)) 

■(ti,...,t^) ^rt.n. 

As said above, 6 (elems (■ (ti, . . . , t„))) C Derw ({5 (■ (ti, . . . , tm))}); 
and as 6 (elems (ti)) C 5 (elems (■ (ti, . . . , tm))), we can (DY)-derive (by 
composition rules) 6 (ti) from 6 (elems (ti)). 
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Proposition 14. Given a standard constraint system S and its normalized 
model a in DY+ACI. Then, for any suhterm of the system t G Sub {S), we 
have 5 (ta) = t6 (cr). 

Proof. The proof is done by induction as in Proposition 5. 

• Let sizeDAG (t) = 1- Tlien eitlier t E A or t E X. Botli are trivial cases. 

• Assume that for some > 1 if sizcDAG {t) ^ k, then 6 {ta) = t6 (cr). 

• Show, that for t such that sizcDAG (^) > ^ + 1, where t = bin (p, g) 
or t = priv (p) and sizeoAG (p) < k and sizcDAG (?) < k, statement 
6 {ta) = t6 {a) is still true. We have: 

— either t = bin [p, q). As 6 (bin [p, q) a) = 6 (bin {pa, qa)) = 
bin {6 (pa) , 6 (qa)) = bin {p6 (a) , q6 (cr)) = bin {p, q) 6 (cr). 

— OT t = priv {p). In this case the proof can be done by analogy with 
previous one. 

Remark: as S is standard, t ^ ■ (L). 

□ 

Theorem 3. Given a standard constraint system S = {Ei > „ and 

its normalized model a in DY+ACI. Then 6 (a) is a model in DY of S. 

Proof. Let E > t he any element of S. As cr is a model of S, then '~ta~^ G 
Der {'~Ea~^). As a is normalized and S is standard, using Lemma 20 we have 
^ta~^ = ta and ^Ea^ = Ea. Then, ta G Der (Ea). That means, there exists 
a DY+ACI derivation D = {Aq, . . . , Ak} such that Aq = Ea and ta G Ak. 

By Lemma 21 and Lemma 3 (which also works for DY case) we can easily 
prove that if /c > 0, S (Aj) C Der^Y {S (Aj^i)) , j = l,...,k. Note, that 
S (A) is a set of standard terms (and thus, normalized) for any set of terms 
A. Then, applying transitivity of Der^^y (■) (Lemma 2 for DY) k times, we 
have that 6 {A^) C Der/jy {6 (Aq)). In the case where k = 0, the statement 
S (Ak) C Der^y {S {Aq)) is also true. 

Using Proposition 14 we have that 6 (Aq) = 6 (Ea) = E6{a), as E (1 
QSub {S). The same for t: 6 {ta) = t6 (cr), and as ta E A^, we have t6 (a) G 

H^k). 
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Thus, we have that tS (a) G S (A^) C DeiDY {S (Aq)) = Der^y {ES (cr)), 
that means 6 (a) DY-satisfies any constraint of S. 

□ 

We present an example iUustrating the theorem. 

Example 8. Let us consider a standard constraint system similar to one in 
Example 5. 

^ _ j 6nc (x, a) , pair (c, a) > 6 1 
[ pair (a;, c) > a j ' 

Using Algorithm 2, we can get a model of S within DY+ACI, let's say, as in 
Example 6, a = {x ^ ■ {{a, b, c})}. 

Then, by applying transformation S {■), we will get a' = 6 (a) = 
{x I— 7- pair (a, pair (6, c))}. We can see, that a' is also a model of S within 
DY(as it was proven in Theorem 3). 

Corollary 3 (of Theorem 3 and Proposition 13). A standard constraint 
system S is satisfiable within DY iff it is satisfiable within DY+ACI. 

Corollary 4. Satisfiability of constraint system within DY is in NP. 
6. Conclusions 

In this work we presented a decision algorithm of satisfiability of general 
constraint system within Dolev-Yao deduction system as well as one extended 
with ACI symbol that can be used to represent sets of terms. The complexity 
class of the algorithm was proved to be in A^P-complete. 

We have given also two applications of the presented result: protocol 
insecurity with non-communicating intruders and discovering XML-based 
attacks. 

APPENDIX 

Appendix A. General constraints for subterm theories 

• composition rules: for all public functional symbols /, Xi, . . . ,Xk 

/(Xi, ...,Xk) 

• decomposition rules: ti, . . . ,tm s, where s is a subterm of tj for some 
i. 
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We show that the satisfiabihty of constraint system within subterm de- 
duction system is undecidable in general. More precisely: 

Instance: a subterm deduction system D, a constraint system C. 

Question: is C satisfiable ? 

To show this, wc reduce the halting problem of a Deterministic Turing 
Machine (TM) M that works on a single tape. We consider the tape alphabet 
r = {0, 1, b}, and b is the blank symbol. The states of the TM M are in a 
finite set Q = {qi, q2, . . . , Qn}- W.l.o.g. we can assume that qi (resp. g„) is 
the unique initial (resp. accepting) state. 

In order to represent Turing machine configuration as terms we shall 
introduce a set of variables X and an alphabet J-" 

{o,i,b,±}ug, 

where {_L} are public functional symbols. 

The TM configuration with tape _L abode _L, (where _L is an endmarker), 
with symbol d under the head, and state q will be represented by the following 
term of q{c{b{a{l.), d{e{l.),x) where x & X and a, b, c,d,e & {0, 1, b}. 

The composition rules we consider for the TM are u f{u) for each 
/ G {0, 1, b} and u,v,w ^ q{u, v, w) for each q E Q. For each TM transition of 
M we will introduce some decomposition deduction rule that can be applied on 
a term representation q{u, v, q'{u', v', x')) iff the transition can be applied to a 
configuration represented by q{u, v, _) and generate a configuration represented 
by q'{u',v', _). For each TM instruction of type: "In state q reading a go to 

state q' and write b", we define the following rule for a,b e {0, 1, b}: 

q{u, a{v), q'{u, b{v),x)) — )■ q'{u, b{v),x) 

For each instruction of type: "In state q reading a go to state q' and move 
right", we define the following rules for a e {0, 1, b} : 

q{u, a{v), q'{a{u),v, x)) — >■ q'{a{u),v, x) 

A rule is for extending the tape on the right when needed: 

q{u, ±, q'{\){u), ±, x)) q'{\){u), ±, x) 
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For each instruction of type: "In state q reading a go to state q' and move 
left", we define the following rules for a G {0, 1, b} : 

q{a{u), V, q'{u, a{v), x)) — )■ q'{u, a{v),x) 
A rule is for extending the tape on the left when needed: 

g(±, a{v), q'{±, \){a{v)),x)) q'{±, \){a{v)),x) 

The resulting deduction system is obviously a subterm deduction 
system. 

Let us consider a constraint S to be solved modulo Dm- 

{gi(±,±,a;)} > qniy,z,w) 

This constraint is satisfiable iff there is a sequence of transitions of M 
from a configuration with initial state qi and empty tape to a configuration 
with an accepting state. Hence the constraint solving problem is undecidable. 

Let us recall the definition of some properties of constraint systems. These 
two properties are natural for modeling standard security protocols: 

variable origination: Vx G Vars (Ei) 3j < i x G Vars (tj), 
monotonicity: j < i =^ Ej C Ei. 

Note that {{gi(±, -i-,x)} > qn{y, z, w)} is obviously monotonic. 

As a consequence, satisfiability of monotonic constraint systems (but 
without variable origination) is undecidable. Here is another constraint 
system, where variable origination is satisfied, but monotony is not. It can 
be used for reducing the halting problem again: 

{{±}l>x, {qi{J., J.,x)} > qniy, z,w)} 

As a consequence, satisfiability of constraint systems with variable origi- 
nation (but without monotonicity) is undecidable. 

We should note by contrast (see [13]), that constraint solving in subterm 
convergent theories is decidable if the constraint system S = {Ei t> ti}-^-^ ^ 
satisfies both variable origination and monotonicity. 
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Appendix B. Proofs 

Appendix B.l. Proofs of several statements of Lemma 4 

Statement 1: Follows from the definition of the normalization function and 
Definition 3.4. 

Statement 3: By induction on sIzcdag (s). Let us fix t. 

• sizeDAG (-s) = sizcDAG ('"i"')- Then s = '~t~^ and '~s~^ = '"'"t"'"', and 
from Statement 2 (by taking empty a) we have '"t"' = ^r^nn^ and 
thus '"s"' = s. 

• Suppose, that for some k, for any s G QSub('~t~'), such that 
sizcDAG (s) > k, s = '"s"'. 

• Consider case, where s G QSub ('"t"') and sizeoAG (s) = k. Then, 
by definition of QSub (■), s is in 

— priv (s) G QSub(rt"i). By induction supposition we have 
'"priv (s)"! = priv (s), and as ^priv {s)~^ = priv ('"s"'), we have 
s = ^s^. 

— bin(s,p) G QSub(rt"i). By induction we have '~bin(s,p)"' = 
bin(s,p), and as '~bin(s,p)~' = bin (^s^ , ^pi), we have s = 

— bin (p, s) G QSub ('"t"'). The similar case. 

— s G elems(- (L)),- (L) G QSub(ri(:n). As sIzcdag (■ (-^^)) > k, 
we have ■ (L) = ^- {L)~^, that means (from Definition 3.7), that 
L is a list of normalized non-ACI-set terms, and as elems (L) ^ 
L, we have that s is normalized. 

Statement 4: Suppose the opposite and let us take s G Sub ('"t"') with 
maximal sizeoAG (s) that does not satisfy the desired property. Note 
that the "biggest" term in Sub ('"t"'), i.e. '"t"', does satisfy the property, 
as we can choose s' = t E Sub (t). By definition of Sub (■) if s G Sub ('"t"') 
and s 7^ '"t"' then there exists r G Sub {'~t~^) such that 

• r = bin (p, s) or r = bin(s,p) or r = priv(s). Without loss 
of generality we consider only the first case (r = bin(p, s)) as 
other ones are similar. As sIzcdag ('") > sIzcdag (s), there exists 
r' G Sub (t) such that r = '~r'~'. By definition of 
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— either r' = bin {p', s') and '~p'^ = p and '~s'^ = s. As s' G 
Sub (r') C Sub (t) the property is proved. 

— or r' = ■ (L) and relems (L)^ = {r}. Since for all q G elems (L), 
root (g) 7^ then there exists q G elems (L) such that q = 
bin (p', s') and i"/)'"! = p and = s. Using Statement 17 we 
have s' G Sub (t). 

• r = ■ (L) and s E L. Then, (since sIzcdag ij) > sIzcdag (s)) 
there exists r' G Sub (t) such that '~r'~' = r. Using Lemma re- 
flemma:DAGvsQuasi and Statement 3 we obtain r — normalized, 
and thus, root (s) 7^ ■. Then by definition of we have r' = ■ [L') 
and L ^ '"elems (L')"!, and thus, s G '"elems (L')"!, that is there 
exists s' G elems (L') such that s = '~s'~'. Using again Statement 17 
we have s' G Sub (t). 

Statement 7: To get the first part of equality we first use Statement 5: 
elems (r- (rt^n , . . . , ^tm^)^) = ^elems (■ (rt^n , . . . , rt„n))n; then from 
Definition 3.4 and Statement 5 we have that elems (■ ('"ti"' , . . . , '"^m"')) 
is a set of normalized terms. The second part directly follows from 
Definition 3.4. 

Statement 8: By induction on sizeoAG (^)- 

• sizeoAG (t) = 1; impUes t = a E A and then elems (a) = {a}, i.e. 
the equality becomes trivial. 

• Suppose, that for any t : sIzcdag (^) < k (k > 1), H (t) = 

Up6elcms(t) ^ (p) holds. 

• Given a term t : sIzcdag (t) = k, k > 1. We shold prove H (t) = 

Upeelems(t) ^ ■ 

— t = priv (ti) or t = bin {p, q). In both cases, elems {t) = {t}, 
and thus, he equality is trivial. 

— t = ■ (L). Note, that for all s E L, sIzcdag (s) < k. Then, 
on the one hand, H {■ (L)) = [J^^^H{p) = (by induction 
supposition) = UpeL Up'Gcicms(p) ^ (p')- On the other hand, 

Upeclems(-(L)) ^ (P) = Upeljp/gi elems{p') ^ (P) = 

Up'GL Upeclcms(p') H (p). Thus, H (t) = Upeelems(i) ^ (P) ■ 

Statement 10: This follows from Statements 9, 6, Definition 3.12 and from 
equality rr^nn = rfi (Statement 2). 
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statement 12: QSub (t) C QSub (QSub (t)) is trivial as t G QSub (t). Now 
we prove by induction on sizeDAG (t) that QSub (QSub (t)) C QSub (t) 

• sizeoAG (t) = 1- Then t G AUX. As QSub (t) = {t} the statement 
is trivial. 

• Suppose, that for any t : sizeoAC (t) < k {k > 1), the statement is 
true. 

• Given a term t : sizeoAG (t) = k, k > 1. Let us consider all possible 
cases: 

— t = bin(ti,t2)- By definition QSub (t) = {t} U QSub (ti) U 
QSub (ts). Then, QSub (QSub (t)) = QSub (t) U 

QSub (QSub (ti)) UQSub (QSub (t2)) and, as sIzcdag (ti) < k 
for i = 1,2, by using induction supposition we obtain the 
wanted property. 

— t = priv (ti). Proof is similar to one for the case above. 

— t = ■ (L). Then QSub (t) = {t} U Upeeiems(L) QSub (p). And 

QSub (QSub (t)) = QSub (t) UUpeelems(L) QSub (QSub (p)), 

but since sIzcdag (p) < ^ for such any p we can apply in- 
duction supposition and get IJpeeiems(L) QSub (QSub (p)) = 
aeciems(L) Q Sub (p) = Q Sub (t) \ {t} . Then 
QSub (QSub (t)) = QSub {t) U (QSub (t) \ {t}) = QSub (t). 

Statement 15: By induction on sIzcdag (t) 

• sizeoAG (^) = 1- 

— t ^ A. As ta = t and Vars (t) = 0, the statement becomes 
trivial. 

— t E X. Then Sub (t) a = ta, Vars (t) = {t}; and as for any 
term p, p G Sub (p), we have Sub (ta) = {ta} U Sub (ta). 

• Suppose, that for any t : sIzcdag (t) < k (k > 1), the statement is 
true. 

• Given a term t : sizeDAG (t) = k, k > 1. Let us consider all possible 
cases: 

— t = bin(ti,t2)- Then ta = hm{tia,t2a) and Vars (t) = 
Vars (ti)U Vars (ts). Sub (ta) = {tajuSub (tio-)USub (tsO") = 
(as sizeDAG (ti) < k) = {ta} U Sub (ti) a U Sub (Vars (ti) a) U 
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Sub (ts) cr U Sub (Vars (^2) cr) = {ta} U Sub {ti) a U Sub (ta) cr U 
Sub ((Vars (ti) U Vars (^2))^^) = Sub {t) a U Sub (Vars {t) a). 

— t = priv (ti). Proof is similar to one for the case above. 

— t = ■ ({ti, . . . , tm})- We have ta = ■ ({ticr, . . . , tm<j}) and 
Vars (t) = |Ji=i m Vars (ti). Then we have Sub (ta) = {ta} U 
Ui=i,...,m Sub (tjcr) = (as sizeoAG (U) < k) 

= {ta} U Ui=i,...,m(Sub(ti)aUSub(Vars(ti)(T)) = {ta} U 

U=i,...,™ Sub (t,) a U Sub ( (U=i,...,™ Vars (t,)) a) = 
Sub (t)aUSub (Vars (t) a). 

Statement 16: It foUows from the fact that f(t) = '"t"' and gcr{t) = ta are 
deterministic functions, and thus return at most one value for one given 
argument. 

Statement 17: First we prove that elems (t) C QSub (t). We use induction 
on sizeoAG (^)- 

• If root (t) 7^ •, then elems (t) = {t} C QSub (t). This case includes 
all t such that sIzcdag (t) = 1- Thus we need to consider only 
t = -(L). 

• Suppose that for any t : sIzcdag (t) < k {k > 1), the statement 
holds. 

• If for some t we have sIzcdag (t) = k, k > 1, then elems (t) = 
UpeL^^^^^ (p) QSub(t) = {t} Upei QSub (p). And since 
sizcDAG (p) < k using the induction supposition we obtain the 
wanted statement. 

Now we show that QSub(t) C Sub(t). Again, applying proof by 
induction on on sIzcdag (t) we have: 

• If sizcDAG (t) = 1) then QSub (t) = Sub (t) = {t}. 

• Suppose that for any t : sIzcdag (t) < k (k > 1), the statement 
holds. 

• If for some t we have sIzcdag (t) = k, k > 1, then 

— t = bin (ti,t2) -Then QSub(t) = {t} U QSub(ti) U QSub(t2) 
and Sub (t) = {t} U Sub (ti) U Sub (ta), 

where max{sizeDAG (^1) , sIzcdag (^2)} < k. And then using 
induction supposition we can conclude for this case. 
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— t = priv (ti). Proof is similar to one for the case above. 

— t = ■ ({ti, . . . , tm}). Then we have 

QSub (t) = {t} U Upeeiems({ti,...,t„}) QSub(p) C (using the al- 
ready proved part of the property) 

^ {0 uUpeQSub({ti,...,i„})QSub(p) = (as QSub (QSub (t)) = 
QSub it)) = {t}U|Jpg|j^ QSub (p) C (by induction suppo- 
sition, as sizcDAG {ti) < k for all i) C {t}UlJpg|^^ Sub (p) = 
Sub(t). 

Statement 18: Using Statement 4 and the fact that r."! is a deterministic 
function we obtain Wp, q G Sub ('"t"') p ^ q 3p', q' G Sub (t) : p = 
rp'-\ A g = rq'^ fs^p^ g. And thus, |Sub (rtn)| < |Sub(t)|. 

Statement 19: We have QSub (■ ({ti, . . . , t/})) = {■ ({ti, . . . , t;})} U 
U!=i QSub (elems {U)). Using Statement 17 and 12 we have 
QSub (elems (tj)) C QSub (QSub (tj)) = QSub (tj). Thus, 
QSub (■ ({ti, . . . , tj)) C {. ({ti, . . . , t,})} u QSub (ti) ■ ■ ■ U QSub (t,). 

Appendix B.2. Proof of Property 11 

As was stated before, the measure of the problem input is measure {S) = 
n X sizeoAG {S) + |IE(5)|, where S = {Ei> 

Algorithm 2 returns a normalized proof a for decision problem if it exists. 
Moreover, sIzcdag {xa) < 2 x sizeoAc {S) for any x G Vars (5). 

First, we will normalize Sa. From Lemma 18 follows, that we can do 
it for the time Tn < Pn (measure {Sa)), where Pn is some polynomial with 
non-negative coefficients of some degree m" > 0. 

From the Proposition 12 we will know that check of derivability of a 
normalized ground term g from set of normalized ground terms G takes a 
polynomial time depending on sizeoAG (G U {(7}). That is, there exists a 
polynomial Pg with non-negative coefficients, such that number of operations 
(execution time) to verify the derivability {g from G) will be limited by 
Pg(sizeDAG {G U {g}))- Then the execution time for checking a set of ground 
constraints {Gi > 5'i}i=i,„.,n ^i^^ ^e limited by YJi=i -Pg(sizeDAG {Gi U {gi})). 

To show that the algorithm is in NP we need to show, that execution 
time of check is polynomial limited by measure of algorithm's input, i.e. there 
exists a polynomial P, such that execution time does not exceed 0{P{n x 
sizeoAG ('5) + |IE (5)1)) steps. 
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In our case, execution time T of a check will be T = Tn + T^, where Tg is 
a time needed for checking ground derivability of Sa: Tg < 

-Pg(sizeDAG ('"(-Ei U As Pg is a polynomial, let us say, of 

degree m' > 0, with non-negative coefficients, we can use the fact, that for 
any positive integers Xi, . . . ,Xk we have Yli=i Pg{xi) < Pg{J2i=i ^d- Then 
we have Tg < Pg(^"^j^ sizcDAc ('"(-^i U {tj})cr~i)) and by Statement 18 of 
Lemma 4 we have Tg < Pg(^"^^ sizeoAc ((-E'j U {tj})^)); using the same 
lemma, we have 

Tg < Pg ^sizcDAG {Ei U {U}) + sizeoAG ^U^^^^ 
^ I sizeoAG [Ei) + sizeoAG (ti) + ^ sIzcdag j j < 
<Pg\^ (2sizeDAG {S)) + n X J^(sizeDAG {xa)) < 



i=l 



</>,(2x„x..e„.o(5) + „xX:(2x.i.e..o(5)) 

\ X J 



< 



< Pg(2 X n X sizeoAG (S) + 2 x n x (sIzcdag {<S)Y) < 
<Pg{4xnx (sizeoAG (5))') < (4 x (n x sIzcdag (5) + |E (5)1)^) = 

= O ^(measure (5))^™" 

On the other hand, let us consider Tn. 
We have Tn < Pn(r;, x sIzcdag ('^o") + |E(5cr)|). One can see that the 
number of edges in DAG-representation of Sa (where every variable a; of 5 is 
replaced by xa) will not exceed the number of edges in S plus the number 
of edges of all xa: \E{Sa)\ < |IE(5)| + J2xeVa,rs{s)\^ since a is 

normalized, we can use Lemma 17: Tn < Pn(?2 x sIzcdag i^'^) + |IE(iS)| + 
EaeVars(5)(sizeDAG (xa))^). 

Then, using Lemma 4 (Statement 15) we obtain Sub (Sa) = Sub (S) a U 
Sub (Vars (S) a), and thus, sIzcdag i<Sa) < |Sub (S) o-\+Y,x&Ya.rs{s) sizeDAG (xa) 
From Statement 16 of Lemma 4 follows that |Sub(5)cr| < sIzcdag ('5). 
Since sIzcdag (^^cr) < 2 x sIzcdag ('5) and |Vars(5)| < sizeDAG('5), we ob- 
tain sizcDAG i'Scr) < sizcDAG {<S) + 2 X (sizcDAG i'S))'^- In the same way, 
I]a;eVars(5)(sizeDAG (xa))'^ < sIzcdag (5) X (2 X sIzcdag (5))^ 
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Therefore, Trn < Prn(n x (sizeoAG (5) + 2 x (sizeoAG (5))^) + |E {S) \ + 
sizeoAG {<S) X (2 X sizeoAc {<S))'^) = O ((measure (5))^"""). 

Thus, T = O ^(measure (S))^"^ + (measure (iS))^™" j that shows, that a 

test of a proof returned by the algorithm takes polynomial time what gives 
us a class of complexity. 

Appendix B.3. Proof of Lemma 12 

Let us consider all the cases of DY+ACI rules: 

• ^1,^2 '"pair (^1,^2)"' We have two cases: 

— 3m G QSub(S') such that '"pair (ti, ^2)"' = ^ua^. Then we have 
Ti{H (rpair (ti, t^V)) = 7t{H (pair (r^^n , rf^i))) = 
7T{{paii{7T{H{rt,^)),n{H{rt,^)))}) = 

rpair {tt{H (ti)), tt{H (^2)))^ and then tt{H (rpair (^1,^2)^)) e 
I)eTi{n{H{t,)),niHih))}). 

— $u G QSub (S) such that '"pair (ti, ^2)"' = '~ua~^. Then (by defini- 
tion, Lemma 4 and Proposition 1) it{H ('"pair (^1,^2)"')) = 

7i{H ('"ti^)Uif (rta^)) e Der (rif (t^) U H (^2)^)- By Proposition 1, 

rH (ti)^ C Der {{tt{H (ti))}) and ^i/ (ts)^ C 

Der {{7r{H (^2))}), then by Lemma 3, (ti)^ U (ts)^ 

C Der({7r(i7 (ti))} U {tt{H (12))}). Now, by applying Lemma 2, 

we have 7r(i7 (rpair (ti, ^2)^)) e Der ({7r(iJ (ti))} U {7r(/7 (^2))})- 

So, in this case 7r{H (r)) e Der ({7r(/7 (h)), 7r{H (k))}). 

• ^i;^2 '"enc (ti, t2)~'- Proof of this case can be done by analogy of 
previous one. 

• {ti, ^2} — > '"aenc (ti, ^2)"^- The same. 

• ti,priv(t2) ^sig{ti,pnv (t2)p. 

— 3m G QSub(S') such that '"sig (ti, priv (^2))"' = '"mct"'. Then 
7t{H (rsig (ti, priv (^2))^)) = vr(i7 (sig (ti, priv (t2)))) = 
7r({sig {wiH {rh^)),n{H (rpriv {h)^)))}) = 

rsig (7r(if (ti)), priv {7r{H (^2))))"^ and then 

7r(i/ (rsig (ti, priv (t2))-')) G Der {{7i{H it^)), 7i{H (priv (t2)))}) 

(as 7r{H (priv (t2))) = priv {tt{H (h)))). 
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— $u E QSub (S) such that '"sig (ti, priv (^2))"' = '~ua^. This case 
can be proved in similar way as done for {ti, — ?■ '"pair (ti, t2)~^. 

• ti,...,tm (ti, . . . , tra)^. On One hand, 7r{H (r. (ti, tm)'^)) = 
7t{H (■ (ti, . . . , tj)) = 7i{H (ti) U---UH (tj) G 

Der {^H (ti) U---UH (t^)^). On the other hand, {Up C 

Dei {{tt {H (ti))}). And thus, by Lemma 3, 7r(if (r- (ti, . . . , e 

Der({7r(if(ti)),...,7r(iJ(U)}). 

• enc (ti, ^2) , '~^2~' — > '"^1^- Here we have to show that n{H {'~ti~^)) is 
derivable from {vr(if (enc (^1,^2))), t^{H ('"^2"'))}- Consider two cases: 

— 3u G QSub (S) such that '"enc (^1,^2)"' = '~u(j~^. Then 

7i{H (enc (ti, ts))) = enc {7r{H {ti)),7r{H (t^))), and 7r{H (rt^n)) = 
7r(i/ (ti)) e Der ({enc (7r(i7 (ti)), 7r(iJ (ts))) , r7r(i7 {^t2^)p}). 

— $u E QSub (S) such that '"enc (^1,^2)"' = '"mo""'. Then 

7r(if (enc (ti, ^2))) = 7r(if (ti) Li H (12)). Using Proposition 1, we 
have (ti) U H (ts)^ C Der ({7r(i7 (enc (ti, ^2)))}), thus 
(by Lemma 4) [tip C Der ({vrji/ (enc (ti, ^2)))})- And then, by 
Proposition 1 we have that tt{H (ti)) E Der ('"iJ (ti)"!). Therefore, 
by Lemma 2, we have 

niH (rtp)) = niH (h)) E Der (niH (enc (ti, ts))))- 

• aenc (^1,^2) , '"priv (^2)"' — ^ '"^i"'- Here we have to show that n^H ('"ti"')) 
is derivable from {ni^H (aenc {ti, ^2))), 7r(if ('"priv (^2)"^))}- Consider two 
cases: 

— 3u E QSub (S) such that '"aenc (ti,t2)~' = '"wcr"'. Then 
7r(i7 (aenc (h, ^2))) = aenc {7i{H (ti)), 7r(i7 (ts))), 

and then n{H (^ti^)) = tt{H (ti)) E 
Der ({aenc (7r(iJ (t,)), 7r{H (ta))) , ^priv (7r(/7 (t2)))^}). 
On the other hand, 7i{H ('"priv (^2)^)) = 7r(/7 (priv (^2))) = 
7r({priv(7r(i7(t2)))}) = ^priv (7r(/J (t2)))^. 

— $u E QSub (5*) such that '"aenc (^1,^2)"' = '~ua~^. Then 

7r(if (aenc(ti,t2))) = vr(i?(ti) U H {t2)). Using Proposition 1, 
we have {h) U H {t2p C Der {{Tr{H (aenc {h, ^2)))}), thus (by 
Lemma 4) ^H{tip C Der ({7r(if (aenc (ti, ^2)))})- And then, by 
Proposition 1 we have that 7r(if (ti)) G Der {'~H (ti)"'). Therefore, 
by Lemma 2, 7r(i/ (^tr)) = 7r(i7 (ti)) E Der (7r(i7 (aenc {h, ta)))). 
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• pair (^1,^2) — ^ '"^i"'- Here, as usual, we consider two cases: 

— 3n G QSub (5*) such that ^pair (^1,^2)"' = ^ua^. Then 
Ti{H (pair (ti, ^2))) = pair ij^iH (ti)), ti^H (^2))) and then 
Tx{H{rti^)) = rn{H{h)y E Der({7r(i/(pair (ti,t2)))}). 

— $u E QSub (S) such that rpair (^1,^2)"' = '~ua~^. Then 

7r(i7 (pair(ti,t2))) = t^{H (ti) U //(ta))- Then by Proposition 1, 
we have (ti) U H (^2)^ C Der {{7i{H (pair (ti, ^2)))}), thus 

(ti)^ C Der {{n{H (pair (ti, ^2)))})- And then, by Proposition 1 
we have that 7i{H (ti)) E Der ('"if (^i)^)- Therefore, by Lemma 2, 
7r(if (rtp)) = n{H (h)) E Der {n{H (pair (ti, t2)))). 

• pair (^1,^2) — > '"^2"'- Proof hke above. 

• ■ (ti, . . . , t„) ^ r^.n. We have 7t{H (■ (ti, . . . , t2))) = n{H (ti) U ■ ■ ■ U 
H (tm)). Then by Proposition 1, (ti) U---UH (tm)^ C 
Der(7r(ii(-(ti,...,t^)))); thus rii(t,)n c Der (7r(ii (■ (ti, . . . , t„)))). 
As 7i{H{ti)) E Der (rif (t^)n), by Lemma 2 we have 7r(if (rt^^)) = 
7r(ii(t,)) eDer(7r(iJ(-(ti,...,t2)))). 

As all possible cases satisfy lemma conditions, we proved the lemma. 
Appendix B.4- Proof of Property 8 

Proof. From proposition 2 and 3 we know that if a' is a model of S then '~cr'~' 
is a model of S and '"a'"' is a model of '"5"'. Then, there exists a substitution 
e : dom (^) = dom {ra'^) , dom (^) 9 C dom (^) , a" = ^a'^ \ dom(6»)6' and a" is 
a model of ^S^O such that xa" 7^ ya", if x 7^ ?/ (this is true because we 
can show how to build 9 : given the '"o"'"' — simply split dom {^(y'~^) into 
the classes of equivalence modulo '~cr'~', i.e. x = y <^==^ x'~a'^ = y^a'^; for 
every class choose one representative [x] = , and then x9 = [x]=). Note, that 
9a" = a', that's why a" is a model of ^S'^ 9. 

Then, as a" is a model of '~S~^9, using Proposition 2, we can say that 
a" is a model of '~'~S~^9~^. Moreover, a" is normalized and xa" 7^ ya" for 
all x,y E dom (cr") such that x ^ y. Then, we can apply Corollary 1, 
which gives us existence of conservative model 6 of '''~S^9^. That is why 
we can apply Proposition 7: for any x E Vars ('"'"5"' 6'"'), sizeoAG (a^^) < 
2 X sizeDAG(^^'5^^^)- 

Note, that using Proposition 2, Lemma 16 and definition of "model", we 
can easily show that 6[9] is a model of ^S^. Moreover, 6[9] is normalized. 
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By definition of S[6] we can say, tliat for all x G dom(5[0]) there exists 
y G dom (6*) 9 such that x6[9] = y6; and as y E X (by definition of 9), then 
sizeoAG {xS[9]) = sizeDAc (yS) < 2 x sizeDAG {''^<S^ 9^) < 2 x sizeDAG {^<S^ 9). 
Applying Lemma 15, we have sizeDAG (^^i^l^]) < 2 x sizeDAG 

Summing up, we have a normalized model a = 6[9] of '"5"' such that for 
all X G dom (a), sizeDAG (^^cr) < 2 x sizeDAG 

□ 
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